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Abstract 


When two incompatible images are contemporaneously displayed to homolo- 
gous portions of the two retinae normal binocular fusion fails, and conscious per- 
ception continuously oscillates between the two alternatives despite the constant ret- 
inal stimulation. This unique form of bistable visual perception is called binocular 
rivalry, and is a peculiar visual phenomenon that engages competition between mo- 
nocular signals and neural representations of the two images at different levels of 
visual processing, from LGN to object-selective infero-temporal cerebral areas. Here 
we present several experiments in which we have used binocular rivalry as a tool to 
investigate different aspects of visual and multisensory perception. 

In the first set of studies we demosntrated that touch specifically interacts with 
vision during binocular rivalry and that the interaction likely occurs at early stages of 
visual processing, probably V1 or V2. We found in fact that touching an engraved 
grating interfered with the dynamics of binocular rivalry both by prolonging domi- 
nance and by curtailing suppression of the congruent visual stimulus (parallel grat- 
ing). We further demonstrated that the interaction between vision and touch during 
binocular rivalry was tightly tuned for matched visuo-haptic spatial frequencies and 
orientations. We finally showed that voluntary attention, action and proprioception 
do not play a leading role in mediating the interaction between vision and touch 
during rivalry, while spatial and temporal proximity between the visual and the hap- 
tic stimuli are necessary conditions for the interaction to occur. 

In the second set of studies we demonstrated that human adult visual cortex re- 
tains an unexpected high degree of experience-dependent plasticity by showing that 
a brief period of monocular deprivation produced important perceptual conse- 
quences on the dynamics of binocular rivalry, causing the previously deprived eye to 
dominate rivalrous visual perception twice as long as the non-deprived eye. We also 
found that the effects of monocular deprivation were more long-lasting when visual 
stimuli modulated in chromaticity compared to luminance-modulated stimuli were 
tested, a result suggesting that chromatic vision retains a higher plastic potential. Fi- 
nally, by showing that monocular deprivation also resulted in stimuli presented to 
the deprived eye appearing higher in contrast compared to stimuli presented to the 
non-deprived eye, we suggest that the perceptual advantage of the deprived eye dur- 
ing binocular rivalry measured after a short period of monocular deprivation reflects 
a compensatory homeostatic up-regulation of contrast gain control mechanisms that 
could be the first reaction of the visual system to the lack of information caused by 
deprivation. 
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In conclusion, the work presented here demonstrates that, because of its unique 
characteristics binocular rivalry is a poverfool tool to investigate different aspects of 
visual perception and can be used to reveal unexpected properties of early visual cor- 
tex. We finally propose that, as suggested by preliminary results, binocular rivalry 
could be used as a non-invasive tool to monitor neuroplasticity during the recovery 
of vision in amblyopic children. 


Chapter 1 
General Introduction 


1.1. What is Binocular Rivalry? 


Our visual system is often faced with perceptual ambiguity and perceptual deci- 
sions need to be made in favour of one of the different interpretations to efficiently 
interact with the external world. According to the Bayesian theory of perception (for 
review see Knill & Pouget, 2004) the brain deals with perceptual uncertainty and 
ambiguity by representing sensory information in form of probability distributions. 
If different perceptual interpretations have the same likelihood and are mutually ex- 
clusive, the visual system cannot “decide” in favour of one or the other and visual 
perception continuously oscillates between the two alternatives. Most of the times 
ambiguous figures producing perceptual alternations (a behavior called bistability) 
are a consequence of unnatural visual stimulation often caused by a failure in depth 
perception derived by mapping a three-dimensional object into a flat surface. Bista- 
ble perception (Blake & Logothetis, 2002) is thought to reflect the competition be- 
tween population of neurons encoding the two incompatible interpretations. 

Ambiguous figures have been rousing the interest of visual scientists and artists. 
Figure 1.1.1 reports the most famous examples of bistable figures taking advantage 
of different forms of ambiguity: ambiguity in depth (the Necker cube, Necker, 1832, 
the Koffka cube, Koffka, 1935, and Schroeder’s stairs), ambiguity in figure-ground 
segregation (Rubin’s face-vase illusion, Rubin, 1915) and ambiguity between high- 
level interpretations of images (Boring’s young girl/old woman figure, Boring, 1930 
and Jastrow’s rabbit/duck image, Jastrow, 1899). The lower panels of figure 1.1.1. re- 
port two examples of how ambiguous figures have been used by artists creating 
stunning pictorial effects. 

Binocular rivalry is a particular type of perceptual bistability that is caused by a 
conflict between monocular images rather than between different interpretations of 
the same monocular image. When two incompatible visual stimuli are contempora- 
neously displayed on homologous portions of the two retinae the brain lapses into 
confusion, interpreting the two visual signals as arising from two different objects 
occupying the same spatio-temporal location in the external world. As a result, the 
two images do not merge into a coherent percept, but engage a strong competition 
for visual awareness with visual perception continuously alternating between the two 
rivalring visual stimuli (a diagram of the phenomenon is shown in Figure 1.1.3.). 
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The systematic study of binocular rivalry was initiated by Sir Charles Wheat- 
stone, who in 1838 invented a simple and smart instrument that allowed the con- 
temporaneous presentation of different visual stimuli to the eyes (Wheatstone, 
1838): the mirror stereoscope (Figure 1.1.2). After characterizing stereo-depth created 
using the mirror stereoscope, Wheatstone noted that something funny happened 
when dissimilar images were presented to the eyes (for example two different letters 
surrounded by a common circle to promote binocular fusion). 


6 si Seng o 

E ` Ey a 

Figure 1.1.1. Examples of ambiguous figures leading to bistable perception. 
Different types of well known ambiguous figures (Panels A-F) are reported: (A) the Necker 
cube (Necker, 1832), (B) the Koffka cube (Koffka, 1935) and (C) Shroeder’s stair, for ambi- 
guity in depth; (D) Rubin’s vase/face illusion (Rubin, 1915) for ambiguity in figure-ground 
segregation; (E) Boring’s old woman/young lady figure (Boring, 1930) and (F) Jastrow’s rab- 
bit/duck figure (Jastrow, 1899). The mutual exclusivity between different interpretations of 
each figure generates continuous perceptual alternations between the two views, a phenome- 
non known as visual bistability. Two examples of artworks using ambiguous figures to create 
visual illusions are reported: (G) M.C. Escher, Convex and Concave, 1955 and (H) J. Albers, 
To Monte Alban, 1942. Ambiguous figures have been used in art since the roman empire. 
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At the beginning of the nineteenth century, Wheatstone described the character- 
istics of this new, unusual visual phenomenon capturing all the main features of bin- 
ocular rivalry (comprising the most controversial characteristics of binocular rivalry 
that we will discuss in the following paragraphs, as they are still currently debated): 


“If a and b (fig. 25.) are each presented at the same time to a different eye, the 
common border will remain constant, while the letter within it will change alternate- 
ly from that which would be perceived by the right eye alone to that which would be 
perceived by the left eye alone. At the moment of change the letter which has just 
been seen breaks into fragments, while fragments of the letter which is about to ap- 
pear mingle with them, and are immediately after replaced by the entire letter. It 
does not appear to be in the power of the will to determine the appearance of either 
of the letters, but the duration of the appearance seems to depend on causes which 
are under our control: thus if the two pictures be equally illuminated, the alterna- 
tions appear in general of equal duration; but if one picture be in ore illuminated 
than the other, that which is less so will be perceived during a shorter time. I have 
generally made this experiment with the apparatus, fig. 6. When complex pictures 
are employed in the stereoscope, various parts of them alternate differently.” 
(Wheatstone, 1838), p.383) 


Figure 1.1.2. The mirror stereoscope invented by Sir Charles Wheatstone. 
Adapted from (Wheatstone, 1838) 


After the observations made by Wheatstone, binocular rivalry has attracted the 
attention of a growing number of visual scientists and is now one of the “hot topics” 
in visual research (D. H. Baker, 2010) because of the peculiar properties that make 
binocular rivalry a fascinating and unique phenomenon and an extraordinary tool to 
investigate different aspects of visual perception (Figure 1.1.4. shows a histogram of 
the number of publications per year regarding binocular rivalry over the past sixty 
years). 
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What is striking about binocular rivalry is that at any time one of the competing 
monocualr images, albeit present on the retina, is rendered invisible, a phenomenon 
known as binocular rivalry suppression. During binocular rivalry each of the two im- 
ages, in fact, undergoes several phases of dominance and suppression with a peculiar 
dynamics that is linked to the relative strength of the visual signals associated with 
each image (Levelt, 1965). Dominance and suppression phases during binocular ri- 
valry are thought to be independent processes (Blake & Logothetis, 2002) and show 
different sensitivities to external influences (e.g. attention, visual context, adapta- 
tion). Binocular rivalry suppression will be discussed later in a dedicated paragraph. 

Because of the atypical dissociation between fluctuating visual conscious percep- 
tion and constant physical stimulation, binocular rivalry has been expressively ascer- 
tained as one of the most interesting methods for investigating the neural correlates 
of visual awareness. The idea of taking advantage of bistable perception in general 
and binocular rivalry in particular to investigate at which stage of neural processing 
visual awareness (and consciousness) emerges has been influentially propounded in 
1998 by Crick and Koch, whose paper “Consciousness and Neuroscience” (Crick & 
Koch, 1998) reviewed electrophysiological experiments on monkeys perceiving bin- 
ocular rivalry (Leopold & Logothetis, 1996; Logothetis & Schall, 1989; Sheinberg & 
Logothetis, 1997) capturing the attention of a vast audience and giving rise to the 
“binocular rivalry explosion” (D. H. Baker, 2010) (Figure 1.1.3). 


Binocular Rivalry 


physical 
stimulation 


conscious 
perception 


Figure 1.1.3. Binocular Rivalry 

Diagram representing the phenomenon of binocular rivalry: when two unrelated images (for 
example orthogonal gratings) are contemporaneously presented to the eyes visual conscious 
perception alternates between the two images despite constant physical stimulation. 
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Figure 1.1.4. Number of publications on binocular rivalry (past 60 years). Adapted from (D. 
H. Baker, 2010) 

Histogram of the number of papers indexed by PubMed on Binocular Rivalry by year. The 
arrows indicate the publication of the initial report by Levelt aand the publication of the in- 
fluential paper on consciousness by Crick & Koch. 


1.2. What rivals during binocular rivalry? Eye versus Stimulus Rivalry. 


Binocular rivalry is a unique and complex visual phenomenon: the question 
about the neural correlates underlying binocular rivalry has generated a widespread 
debate and a lot of experimental evidence has been produced in favour of two major 
theories, one supporting the idea of binocular rivalry being generated at early stages 
of visual processing from the conflict between monocular representations of the 
competing stimuli (eye rivalry), the other supporting the idea of binocular rivalry 
being generated between high-level representations of the competing images pro- 
cessed at higher levels of visual analysis (stimulus rivalry). 

The theory of eye-based rivalry has been formulated in the late 80s and consid- 
ered binocular rivalry as being a local process, originating from the competition be- 
tween neuronal populations representing corresponding regions of the two eyes, the 
population of neurons receiving the input from the dominant eye suppressing the 
corresponding neural population of the other eye. In this view, binocular rivalry is 
thought to emerge early in the visual system, where the neural populations repre- 
senting the rivalring stimuli preserve the information about the eye-of-origin, so be- 
fore binocular combination (Blake, 1989; Lehky, 1988). Consequently, according to 
this model, binocular rivalry derives from interocular competition at the thalamic 
level (LGN) and in the primary visual cortex, that is the last stage of visual analysis 
receiving direct inputs from monocular neurons (Hubel & Wiesel, 1977; Maunsell & 
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Van Essen, 1983). One of the main psychophysical evidence supporting eye-rivalry 
at that time was the non-selectivity of binocular rivalry suppression: during suppres- 
sion phases of binocular rivalry, sensitivity markedly decreases, but this lack of sensi- 
tivity is generalized to all information presented to the suppressed eye and not lim- 
ited to the features regarding the suppressed stimulus. Non-selectivity implies that 
suppression is acting on all stimulus features, say at a low-level, for a local portion of 
the eye (Blake & Fox, 1974c; Fox & Check, 1968; O'Shea & Crassini, 1981b; Wales & 
Fox, 1970; Zimba & Blake, 1983). 

More recently, eye-rivalry has been supported by psychophysical evidence show- 
ing that eye-of-origin information is important for different aspects of binocular ri- 
valry: dominance during binocular rivalry between complex images (faces) has been 
shown to spread from monocular regions (Arnold, James, & Roseboom, 2009), bin- 
ocular rivalry has been shown to be sensitive to real-world occlusion constraints 
linked to monocular information (Shimojo & Nakayama, 1990), exogenous attention 
can be attracted during binocular rivalry by presenting a monocular cue (Ooi & He, 
1999) and the dominant eye has been show to stabilize rivalrous perception in the 
initial seconds of dominance (Bartels & Logothetis, 2010). Further support to eye- 
based rivalry has been obtained by electrophysiological recordings showing strong 
interocular suppression in the primary visual cortex of cats during dichoptic stimu- 
lation (Sengpiel, Blakemore, Kind, & Harrad, 1994) and by neuroimaging studies 
showing awareness-related modulations of BOLD signal during binocular rivalry at 
early levels of visual processing: LGN (Haynes, Deichmann, & Rees, 2005; Wunder- 
lich, Schneider, & Kastner, 2005) and monocular regions of primary visual cortex 
identified using the representation of the blind spot (Tong & Engel, 2001). 

The theory contradicting eye-based models of binocular rivalry has been derived 
by two main experiments both published in 1996, one showing interocular grouping 
between rivalring visual stimuli (Kovacs, Papathomas, Yang, & Feher, 1996), the 
other demonstrating the survival of slow perceptual alternations despite the contin- 
uous interocular swapping of the two rivalring images (Logothetis, Leopold, & 
Sheinberg, 1996). 

With a simple and smart paradigm, Kovacs et al (1996) tackled the question 
whether monocular representations or stimulus representations were engaged in 
binocular rivalry by presenting different patchwork images to the eyes each one be- 
ing made of parts of two coherent images (i.e. a monkey and a jungle with a writing, 
Figure 1.2.1B). If binocular rivalry were the result of competition between monocu- 
lar signals visual perception would be expected to oscillate between the two patch- 
work images, if instead rivalry resulted from the competition between coherent rep- 
resentations of complex images, perception would be expected to alternate between 
the monkey face and the writing (Kovacs, Papathomas, Yang, & Feher, 1996). To re- 
duce the influence of semantic processing and isolate the role of pattern coherency 
in guiding binocular rivalry alternations, the authors focused on chromaticity as 
grouping attribute and presented pairs of visual stimuli made of patchwork colors 
(red and green over a yellow background, Figure 1.2.1D). Kovacs et al (1996) found 
that rivalry between uniform colors was perceived on 60% of trials, indicating inter- 
ocular grouping between coherent elements during binocular rivalry. From this re- 
sult the authors concluded that binocular rivalry occurred between stimulus repre- 
sentations, driven by pattern coherency rather than being only driven by monocular 
inhibitory interactions (Kovacs, Papathomas, Yang, & Feher, 1996). 
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Figure 1.2.1. interocular grouping during binocular rivalry. Adapted from (Kovacs, Pa- 
pathomas, Yang, & Feher, 1996) 

Visual stimuli used by Kovacs et al to demonstrate the occurrence of interocular grouping 
during binocular rivalry, when patchworks of two coherent images are presented dichoptical- 
ly, visual perception alternates between the two coherent patterns most of the time. 


Inspired by this first result, several successive experiments have been performed 
demonstrating interocular grouping during binocular rivalry, including contour 
grouping (“association field”, Alais, Lorenceau, Arrighi, & Cass, 2006), interocular 
grouping during different forms of rivalry (Pearson & Clifford, 2005b) and flicker- 
modulated grouping (Knapen, Paffen, Kanai, & van Ee, 2007; Silver & Logothetis, 
2007). Support to the involvement of top-down feedbacks in regulating visual per- 
ception during binocular rivalry (being probably involved in mediating interocular 
grouping), has been also obtained by neuroimaging studies, demonstrating the acti- 
vation of a fronto-parietal network (also implicated in spatial attention) of cortical 
areas during binocular rivalry (Lumer, Friston, & Rees, 1998). 

The other strong evidence in favour of stimulus-based rivalry view was present- 
ed by Logothetis, Leopold and Sheinberg (1996) that introduced an interocular 
switching paradigm (IOS), in which orthogonal gratings flickering at a frequency 18 
Hz were swapped between the eyes every 333 milliseconds (Figure 1.2.2.). If binocu- 
lar rivalry was guided only by reciprocal inhibition between monocular representa- 
tions, visual perception would alternate regularly following the interocular swapping 
rate (3 Hz). The authors found instead that, under these conditions, observers expe- 
rienced the classic slow irregular alternations that characterize binocular rivalry, in- 
dicating that binocular rivalry also arises from competition between high level repre- 
sentations of the stimuli. The authors named this phenomenon stimulus rivalry, 
(Logothetis, Leopold, & Sheinberg, 1996). 
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The currently most common conceptualization of binocular rivalry is a hybrid 
model in which binocular rivalry is thought to take place at different levels of visual 
analysis, from different and interconnected neural events distributed over the visual 
hierarchy (Alais & Blake, 2005; Blake, 2001; Blake & Logothetis, 2002; Tong, Meng, 
& Blake, 2006; H. R. Wilson, 2003). Eye-based and stimulus-based rivalry have in 
fact been demonstrated to be different, but related phenomena, coexisting, rather 
than being exclusive. For example, Lee and Blake (1999) have shown that stimulus 
rivalry occurs only under specific, controlled conditions, and is confined to a limited 
range of spatial and temporal frequencies (Lee & Blake, 1999); a couple of years later 
Bonneh, Sagi and Karni (2001) have extended the specificity of stimulus rivalry to 
stimulus coherence, size and contrast (Bonneh, Sagi, & Karni, 2001). 


18 Hz 


“MLT, SIMIL 


Stimulus flicker 


Stimulus configuration 


Non-reversal condition 


b [ Stimulus 


Swap 
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Stimulus flicker 


Reversal condition 


Figure 1.2.2. Interocular Switching Paradigm (IOS). Adapted from (Logothetis, Leopold, & 
Sheinberg, 1996). 

Orthogonal gratings flickering at 18 Hz presented dichoptically are swapped between the eyes 
every 333 milliseconds. 


Eye- and stimulus-rivalry have also been demonstrated to occur at different times 
during a dominance phase, eye-of-origin information playing a leading role in the ini- 
tial stabilization of visual perception after a perceptual switch during binocular rivalry, 
stimulus configuration contributing importantly in the subsequent perceptual stability 
of the dominant image (regardless of the visual stimulus used, Bartels & Logothetis, 
2010). This has been shown either by swapping the images between the eyes or by in- 
troducing brief blank periods disrupting stimulus coherency: dominance of a particu- 
lar visual stimulus was interrupted when the interocular swap occurred early after a 
switch, but not by early blanks, while a few seconds after dominance stabilization, a 
switch was more likely to be caused by blank periods rather than eye-swaps (Bartels & 
Logothetis, 2010). Finally, a different contribution of the Magnocellular and Parvocel- 
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lular pathways has been recently demonstrated for the two types of rivalry, eye rivalry 
being mostly driven by the Magnocellular pathway and stimulus rivalry being mostly 
driven by the Parvocellular pathway (Denison & Silver, 2012). 
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Figure 1.2.3. Hybrid Model of Binocular Rivalry. Adapted from (Tong, Meng, & Blake, 2006). 
The figure shows the three levels of interaction between visual neurons during rivalry: (a) the 
inhibitory connections between monocular neurons instigating rivalry; (b) Excitatory connec- 
tions within low-level or high-level neurons promoting different types of grouping; (c) Feedback 
projections from high- to low-level areas, modulating neuronal activity at the monocular level. 


Taken together, psychophysical evidence revealing the existence of two distinct 
but related types of rivalry, one eye-based, preserving information about the eye-of 
origin, the other stimulus-based, immune to interocular swaps and showing interocu- 
lar grouping, electrophysiological recordings from monkey brain exhibiting a hierar- 
chy of awareness-dependent fluctuations of neuronal activity and neuroimaging re- 
cordings in humans showing both early (LGN, V1) and higher level (inferotemporal 


17 


Early cross-modal interactions and adult human visual cortical plasticity 


cortex, fronto- parietal areas, attention network) modulations ofthe BOLD signal sup- 
port the hybrid model that characterizes binocular rivalry as being a complex phe- 
nomenon recruiting a distributed network of visual activity. Visual competition during 
binocular rivalry starts early in the visual system, where it is mainly driven by interocu- 
lar competition (i.e. reciprocal inhibition between monocular neural populations). 
Proceeding along the visual hierarchy, after binocular combination, competition goes 
on between neural populations representing visual stimuli configuration. Activity 
from these higher level areas can feed back into early areas to count for interocular 
grouping observed for patchwork monocular images, and frontal and parietal areas 
(neural network also engaged in spatial attention), modulates switching by feedbacks 
to the prior stages of processing, Figure 1.2.3, Tong, Meng, & Blake, 2006). 


1.3. Binocular Rivalry dynamics. 


One of the particular characteristics of binocular rivalry dynamics is the unpre- 
dictability of dominance phase durations. Perceptual oscillations during binocular 
rivalry, in fact, are not regular: periods of dominance of the visual stimulus present- 
ed to the right eye and that presented to the left eye do not follow a stable time con- 
stant and seem to be governed by a stochastic mechanism (Fox & J., 1967; Lehky, 
1995; Levelt, 1966, , 1967). Dominance phase durations during binocular rivalry are 
independent: even considering the whole prior history of phase durations, the dura- 
tion of the current dominant phase would be unpredictable. 

Another feature of binocular rivalry is its ineluctability: once the conditions fa- 
vourable to elicit binocular rivalry are created (i.e. when two images differing along 
one of diverse spatial and temporal dimensions are separately presented to the eyes) 
the observer cannot avoid experiencing perceptual alternations. Not only the induc- 
tion, but also the continuation of binocular rivalry alternations is compulsory: the 
observer cannot hold indefinitely one of the two visual stimuli: no matter its rele- 
vance and meaning (Blake, 1988) or observer’s voluntary attention (Lack, 1978), the 
temporarily dominant image will always be supplanted by the previously suppressed 
image presented to the other eye. 

Finally, binocular rivalry alternations are characterized by mutual exclusivity: one 
of the two rivalring images is perceived at a time only to be replaced by the previously 
suppressed image, in a winner-takes-it-all dynamics and the two visual stimuli are not 
perceived at the same time. Brief periods of “patchy rivalry” can nevertheless occur 
during transitions from dominance of one stimulus to dominance of the other. Per- 
ceptual switches during binocular rivalry are not immediate like a sequence of flashed 
photographs, but they reflect a travelling wave of neural activity (Lee, Blake, & Heeger, 
2005) originating from a region of the visual stimulus and then spreading to the whole 
image in a wave-form way (Paffen, Naber, & Verstraten, 2008). These transitional 
waves are especially evident and slower when large visual stimuli are presented in cen- 
tral vision, suggesting that perceptual dominance during binocular rivalry is mediated 
by local cooperating neural populations that are organized according to retinotopic 
coordinates (Wilson, Blake, & Lee, 2001). Under optimal conditions (small, high- 
contrasted visual stimuli) however, periods of mixed perception do not normally ex- 
ceed 10-15% of the total viewing time and are confined to transitions periods. 
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1.3.1. Phase duration distribution 


As anticipated in the previous paragraph, binocular rivalry phase durations are 
unpredictable and stochastically organized. The distribution of phase durations shows 
a typical unimodal distribution with a positive asymmetrical shape skewed towards 
phase durations longer than the average, a type of distribution that is typical of sto- 
chastic phenomena. Phase duration distribution can be approximated either by a 
gamma distribution (Levelt, 1967) or a log normal distribution (Lehky, 1995). Switch- 
ing rate during binocular rivalry and other forms of perceptual bistability is coherent 
for each observer, but shows a large inter-individual variation (Carter & Pettigrew, 
2003). When averaging across observers, in fact, normalizing individual phase dura- 
tion distributions to individual mean phase duration is a common and necessary pro- 
cedure (Levelt, 1967). Interestingly, interindividual differences in switching rate during 
bistable visual perception have been shown to have a genetic component (52% of vari- 
ability explained in monozygotic twins, Miller et al., 2010; Shannon, Patrick, Jiang, 
Bernat, & He, 2011) and to be linked to differences in brain structure and connectivity 
using magnetic resonance imagning (Genc, Bergmann, Singer, & Kohler, 2011; Genc 
et al., 2011; Kanai, Bahrami, & Rees, 2010; Kleinschmidt, Sterzer, & Rees, 2012). 

In this paragraph we will focus on the gamma distribution because it is the most 
commonly used function to model phase duration distribution in binocular rivalry. 
An example of a gamma distribution of the form g(x) = [\!'x!!/T(r)]exp(-Xx), where T 
is the gamma function, is shown in figure 1.3.1. The gamma distribution is character- 
ized by two parameters: r is the shape parameter and J is the scale parameter (we will 
come back to the meaning of the parameters in the next paragraph), the exponential 
distribution and the chi-squared distribution are two particular cases of the gamma 
distribution. 

Even though the gamma-like shape of phase duration distributions has been con- 
sidered an hallmark of binocular rivalry and bistable perception in general (Carter & 
Pettigrew, 2003; van Ee, 2005), and it has also been used as a validation for adequate 
binocular rivalry dynamics (Leopold & Logothetis, 1996, 1999; Logothetis, Leopold, & 
Sheinberg, 1996), some criticism on the application of gamma distribution for model- 
ling binocular rivalry dynamics has been risen. For example, since the two parameters 
of the gamma distribution usually correlate or are even identical when binocular rival- 
ry phase duration distributions are fitted by a gamma distribution function, the two 
parameters have been considered as redundant (De Marco, Penengo, & Trabucco, 
1977; Mamassian & Goutcher, 2005) and so the gamma function. 

In general, using phase duration distribution as only parameter to describe the 
dynamics of binocular rivalry is limiting, because it does not take into account the 
temporal succession of events during an extended period of binocular rivalry. 
Phase duration is an important factor because it is an index of the stability of bi- 
stable perception: short phases indicate periods of unstable perception in which 
switches occur rapidly, while longer phases indicate periods of more stable percep- 
tion during which the alternation rate is slow. As we will detail in the next para- 
graphs, alternation rate is indicative of stimulus strength and can be also altered by 
internal factors as arousal and attention. Even though it is generally true that phase 
durations during binocular rivalry follow stochastic neural fluctuations, over an 
extended epoch of viewing periods of fast and slow alternations may occur 
(Mamassian & Goutcher, 2005) and such variations in alternation rate are not cap- 
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tured by phase duration distributions where phase durations are collapsed disre- 
garding of their temporal order. 


Number of {, in category 


0 04 08 12 16 20 24 28 32 36 40 
t, (rescaled) 


Figure 1.3.1. Binocular Rivalry Phase Duration Distribution. Adapted from (Levelt, 1967). 
The figure shows the histogram of phase durations of one eye fitted by a gamma distribution 
function of the form g(x) = [A’x"’/T(r)]exp(-Ax). Before plotting, phase durations have been 
normalized to the average phase durations of that eye. Normalization is a common procedure 
in binocular rivalry especially when averaging across observers because mean phase durations 
show a massive inter-individual variance. 


In order to compensate for the incompleteness of phase duration distributions 
in describing the temporal dynamics of binocular rivalry, different methods of 
analysis have been proposed, for example, Mamassian & Goutcher (2005) have in- 
troduced the analysis of instantaneous probability as a dynamic method for char- 
acterizing visual perception during binocular rivalry. To improve the objectivity of 
the measurement of visual rivalrous perception they have also introduced a new 
recording method in which the observer is required to report visual perception 
every couple of second in correspondence of an acoustic signal (Mamassian & 
Goutcher, 2005). They have then computed the reversal probability separately for 
the two visual stimuli and the survival probability at each time bin during extend- 
ed periods of observation obtaining “probability traces” describing the dynamics 
of binocular rivalry. 


1.3.2. Stimulus strength and the dynamics of binocular rivalry 


Despite the overall randomness underlying binocular rivalry dynamics, predomi- 
nance of rivalring visual stimuli is dependent on the relative strength of the competing 
images. The dependence of phase duration on stimulus strength has been detailed for the 
first time by W.J. Levelt that in 1965 described how increasing “contour strength” of the 
stimulus presented to one eye increased the overall predominance (percentage of the to- 
tal time of perceptual dominance over an extended period of viewing) of that stimulus 
(Levelt’s first proposition, Levelt, 1965). The observation that the stronger rivalring stim- 
ulus dominates over the weaker one is not surprising and has been confirmed for differ- 
ent low-level characteristics of the visual stimuli: brightness (Fox & Rasche, 1969; I. T. 
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Kaplan & Metlay, 1964), contrast (Hollins, 1980; Whittle, 1965), contour density (Levelt, 
1965), spatial frequency (Andrews & Purves, 1997; Fahle, 1982) and velocity (Blake, Yu, 
Lokey, & Norman, 1998; Wade, de Weert, & Swanston, 1984). 

What happens if stimulus strength is varied symmetrically in both eyes? Levelt’s 
third and fourth propositions address this issue, stating that increasing stimulus 
strength of both visual stimuli during binocular rivalry speeds up the switching rate 
(Levelt, 1966). As for the first one Levelt’s third and fourth propositions have been 
generally accepted by the scientific audience and no objections have been risen 
against them. Levelt’s second proposition is less intuitive and more problematic than 
the other three, stating that the overall predominance of the stronger stimulus is not 
explained by the stronger stimulus showing prolonged dominance durations, but is 
instead due to dominance durations of the weaker stimulus being curtailed. In short, 
increasing stimulus strength in one eye does not increase dominance of that eye, but 
rather decreases its suppression. As a consequence of Levelt’s second proposition, 
counter-intuitively, changes in dominance duration of one eye depend on stimulus 
strength of the other eye. Regarding phase duration distributions, Levelt associated 
the scale parameter A to the strength of the stimulus presented to the other eye, and 
therefore inversely correlated with dominance duration of that eye. 

Even though Levelt’s second proposition has been generally confirmed (Blake, 
1977; Fox & Rasche, 1969; Logothetis, Leopold, & Sheinberg, 1996; Meng & Tong, 
2004), some studies have demonstrated both diminished suppression and prolonged 
dominance durations of one of the rivalring stimuli after increasing its stimulus 
strength (Bossink, Stalmeier, & De Weert, 1993; Brascamp, van Ee, Noest, Jacobs, & 
van den Berg, 2006; Mueller & Blake, 1989), violating one part of the proposition. Dif- 
ferent limiting factors have been proposed to explain the infringement of the proposi- 
tion: the role of periods of mixed perception (Mueller & Blake, 1989), the range of con- 
trasts tested (Brascamp, van Ee, Noest, Jacobs, & van den Berg, 2006) and the role of 
return transitions (cases in which after a period of mixed perception dominance of the 
initial stimulus is re-establised, Bossink, Stalmeier, & De Weert, 1993). 

In a recent study (Kang, 2009), evidence confirming and invalidating Levelt’s second 
proposition has been reviewed and re-considered introducing new methods for testing 
binocular rivalry in order to elucidate the debate on its validity. Using new tracking 
methods, periods of mixed perception and return transition have been shown to be inef- 
fective in explaining prolonged dominance of the stronger stimulus (in this study stimu- 
lus strength was manipulated by increasing contrast in one eye), while stimulus size has 
been proved to be crucial (Kang, 2009). When large visual stimuli were tested Levelt’s 
second proposition was confirmed, while it was violated when small visual stimuli (sub- 
tending less than 1.5 degrees of visual angle) were tested. This finding was then corrobo- 
rated by re-examining the literature and showing that small visual stimuli were used in 
previous studies that had disproved Levelt’s second proposition (Table 1.3.1). 

To clarify the importance of stimulus size in determining the validity of Levelt’s 
second proposition Kang referred to a well-known local component of binocular rival- 
ry (Alais, Lorenceau, Arrighi, & Cass, 2006; Blake, O'Shea, & Mueller, 1992) and pro- 
posed a model in which cooperation between local networks encoding small neigh- 
bouring areas of the visual field explained the three possible perceptual states during 
binocular rivalry (dominance of one or the other visual stimulus and periods of mixed 
perception). Activity of only one local detector violates the contrast-invariance envis- 
aged by Levelt, but the combined output of the modelled local interacting networks 
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replicates the predictions depicted by Levelt’s second proposition. A slightly modified 
version of this model has recently been proposed by the same author (Kang & Blake, 
2011) as a solution to integrate models of binocular rivalry spatio-temporal dynamics 
based on mutual inhibition between the eyes/stimuli, adaptation and noise. According 
to these authors rather than excluding each other, balance between these three phe- 
nomena would underlie the dynamics of binocular rivalry (Kang & Blake, 2011). 


Study Stimulus Size Result 
(Levelt, 1965) reversed 6.00° O 
(Fox & Rasche, 1969) luminance 3.24° O 
(Bossink, Stalmeier, & De Weert, 1993) contrast 1.32° X 
(Meng & Tong, 2004) 6°x2° |O 
(Logothetis, Leopold, & Sheinberg, 1996) Sine 3.00° | O 
(Blake, 1977) wave 1.25° O 
(Mueller & Blake, 1989) grating 0.80° | x 
(Brascamp, van Ee, Noest, Jacobs, & van den Berg, 2006) 0.62° x 


Table 1.3.1. Summary of studies on Levelt’s second proposition. Adapted from (M. S. Kang, 2009). 

The result column indicates with O studies that validated the proposition and with X studies 
violating the proposition. Generally, studies confirming the proposition used large visual 
stimuli, whereas studies denying the proposition used small visual stimuli. 


1.3.3 Onset Rivalry 


As detailed in the previous paragraph, traditionally binocular rivalry dynamics 
have been described through the analysis of phase durations (average phase durations 
and distributions). As anticipated, the analysis of phase durations is important, for 
phase duration is a good index of binocular rivalry stability, with short phase durations 
indicating highly unstable visual perception with a very fast switching rate, and long 
phase durations indicating more stable visual perception with slow alternation rate 
and longer periods of constant perception of one or the other visual stimuli. Switching 
rate is also important because of its relation with stimulus strength, with rivalring visu- 
al stimuli eliciting a strong visual signal alternating more rapidly and weaker visual 
signals alternating slowly (Levelt, 1966, 1967). However, the analysis of phase dura- 
tions is a collapsed measure of binocular rivalry dynamics that does not take into ac- 
count the temporal order of events during a prolonged epochs of viewing. 

Until recently, phase durations analysis was accepted as a standard measure, be- 
cause following fusion of the dichoptic images binocular rivalry dynamics was 
thought to rely on common neural processes. Growing experimental evidence is 
though calling this assumption into question, demonstrating that the properties of 
initial rivalry are substantially different from those of sustained rivalry. Onset rivalry 
shows in fact a high degree of stability and predictability (for review see: Stanley, 
Forte, Cavanagh, & Carter, 2012), as opposed to the randomness shown by sustained 
binocular rivalry perceptual alternations (Fox & J., 1967; Kim, Grabowecky, & Suzu- 
ki, 2006; Lehky, 1995; Levelt, 1965; van Ee, 2009). 

The most striking characteristic exhibited by onset rivalry is a stable bias in fa- 
vour of one of the two dichoptic images that vary across the visual field (Carter & 
Cavanagh, 2007; Stanley, Carter, & Forte, 2011). At one particular location of the 
visual field, in fact the same visual stimulus is perceived at the onset of rivalry (fist 
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coherent percept after binocular fusion) at every presentation. The perceptual bias is 
related to monocular dominance and stimulus contrast, but cannot be entirely ex- 
plained by these two factors (Carter & Cavanagh, 2007; Stanley, Carter, & Forte, 
2011). Furthermore, the onset bias is stable over time (weeks) for every observer and 
disappears when sustained rivalry is tested (Carter & Cavanagh, 2007). 

As reported for sustained rivalry (Levelt, 1967), onset rivalry is influenced by 
stimulus strength, with the stronger visual stimulus being preferred over the weaker 
one (Chong & Blake, 2006) even for small imbalances between the eyes. On the other 
hand, while equating stimulus strength re-establishes perceptual balance between the 
eyes during sustained rivalry (both on total predominance and on mean phase dura- 
tions), the stable bias characterizing onset rivalry resists also when luminance and 
contrast differences between the dichiptically presented visual stimuli is minimized 
for every observer (Stanley, Forte, Cavanagh, & Carter, 2012). 


Table 1 | Studies investigating onset dominance in binocular rivalry. 


Study Onset effect Sustained offect* 


Dominance of the temporal hemifields demonstrated using Not assessed in Crovitz and Lipscomb (1963) 
fulHield color stimuli (Crovitz and Lipscomb, 1963) 


Colored grating patches presented in the periphery caused strong No localized bias observed when brightness was not 
onset biases that varied between individuals and across locations matched (Carter and Cavanagh, 2007). Not assessed in 
within an individual (Carter and Cavanagh, 2007). Areas of temporal Stanley etal. (2011) after brightness matching 
hemifield dominance only became evident after matching each 

location for perceived brightness (Stanley etal., 2011) 


Visual field location 


Retinal image shift renewed onset bias, but more so for stimulus Not assessed as part of image shift experiment 
shifts than saccades (Kalisvaart etal., 2011) (Kalisvaart etal., 2011) 


- aa ee 


Minimizing contrast differences by matching brightness in each Not assessed in Stanley etal. (2011) 
location for each observer unmasked other endogenous biases 
{Stanley etal., 2011) 


> 


ttention 
Effect of attention counteracted by reducing contrast of Not assessed in Chong and Blake (2006) 
attention-boosted target 0.3 log-units (Chong and Blake, 2006) 


Task TAR A spinning Necker cube was more likely to be seen as viewed from Some observers exhibited average dominance bias during 
releva above when rotating on a vertical axis, and from the right when 15-s presentation, though slow switch from biased 
rotating on a horizontal axis on the left side of the screen. Authors dominance at onset may account for this result (Dobbins and 
suggest encoding of ecological relevance (Dobbins and Grossmann, 2010) 
Grossmann, 2010) 


No significant bias toward emotionally arousing images (Sheth and Greater average dominance of emotionally arousing images 
Pham, 2008) after 15 sec of viewing (Sheth and Pham, 2008) 


Oo ES eee oem 


Table 1.3.2. Factors influencing onset and sustained rivalry differently. Adapted from (Stan- 
ley, Forte, Cavanagh, & Carter, 2012). 
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In synthesis, the initial bias in favour of one of the rivalring images seems to be 
guided both by exogenous and endogenous factors and cannot be explained by a 
single feature. Among the factors interacting in influencing onset rivalry are: zones 
of monocular dominance across the visual field (Carter & Cavanagh, 2007; Kalis- 
vaart, Rampersad, & Goossens, 2011), imbalances in stimulus strength (Chong & 
Blake, 2006; Stanley, Carter, & Forte, 2011), both stimulus-driven (Chong & Blake, 
2006; Hancock & Andrews, 2007; J. F. Mitchell, Stoner, & Reynolds, 2004) and vol- 
untary (Chong & Blake, 2006) attention, the position of the visual target on the reti- 
na (Kalisvaart, Rampersad, & Goossens, 2011), learned utility of the visual stimulus 
(Chopin & Mamassian, 2010; Denison, Piazza, & Silver, 2011; Dobbins & Gross- 
mann, 2010) and emotional saliency of the stimulus and observer anxiety level 
(Gray, Adams, & Garner, 2009; Sheth & Pham, 2008). 

Table 1.3.2., adapted from (Stanley, Forte, Cavanagh, & Carter, 2012), reports all fac- 
tors found to influence onset rivalry opposed to the very same factors influence on sus- 
tained rivalry. Interestingly, all of these factors have different effects on onset and sus- 
tained rivalry, suggesting that the two phenomena are guided by different neural mecha- 
nisms. To investigate the dynamics of binocular rivalry appropriately is therefore neces- 
sary to consider both processes, rather than limiting the analysis to phase durations. 


1.4. Binocular Rivalry Suppression 


One of the most fascinating aspects of binocular rivalry is the perceptual invisi- 
bility of a retinal image provoked by binocular rivalry suppression. While one of the 
stimuli dominates observer’s perception the other image is removed from visual 
awareness until a perceptual switch occurs reversing dominance in favour of the 
previously suppressed stimulus. At which stage of neural processing does binocular 
rivalry suppression take place? What is the fate of the suppressed signal? 

A method to quantify binocular rivalry suppression strength consists in measuring 
sensitivity to visual targets briefly presented on one of the rivalring images either dur- 
ing dominance or during suppression phases. The ratio between thresholds measured 
during suppression and those measured during dominance, once subtracted from 1, 
corresponds to an index of suppression depth, where 1 represents complete suppres- 
sion (total invisibility of the visual target for all ranges of contrasts tested) and 0 repre- 
sents no suppression (visual sensitivity during suppression equivalent to that during 
dominance). Surprisingly, despite its phenomenal invisibility, visual sensitivity during 
binocular rivalry suppression is only attenuated of around 0.3 to 0.5 log units, with 
suppression depth being generally comprised between 0.4 and 0.5 (Blake & Fox, 1974c; 
Nguyen, Freeman, & Alais, 2003; O'Shea & Crassini, 1981b; Ooi & Loop, 1994). Meas- 
uring suppression depth is particularly fascinating, for the method mimics “blindsight” 
conditions: under forced choice condition the observer reports a visual target correctly 
despite being consciously unaware of its presence. 

The spared sensitivity found during suppression indicates that information re- 
lated to the suppressed visual stimulus is not completely destroyed despite the total 
perceptual loss (Blake, 1989), demonstrating that a signal that is too weak to reach a 
conscious representation can nevertheless give rise to measurable behavioural effects 
and, consequently, neural activation. Interestingly, suppression depth has been 
shown to increase with stimulus complexity especially for visual stimuli eliciting re- 


24 


Claudia Lunghi 


sponses along the ventral pathway (Alais & Melcher, 2007; Alais & Parker, 2006; 
Blake, 1989; Nguyen, Freeman, & Alais, 2003), suggesting a progressive disruption of 
the suppressed visual signal for different stages of visual analysis. To understand the 
nature and neural organization of binocular rivalry suppression different techniques 
have been used to investigate what can survive suppression, these include adapta- 
tion, priming, influence on the dominant stimulus and neurophysiologic recordings 
(fMRI in humans and electrophysiology in monkeys). 


1.4.1 Visual adaptation and priming to suppressed patterns 


The visual system is continuously adapting adjusting its sensitivity according to 
changes in the external world in order to allow efficient interaction with the envi- 
ronment (Clifford et al., 2007; Kohn, 2007). Adaptation is ubiquitous in the brain, 
occurring at different stages of visual processing, starting from the retina (e.g. light 
and dark adaptation, Shapley & Enroth-Cugell, 1984), to high level temporal visual 
cortices (e.g. face identity adaptation, Leopold, O'Toole, Vetter, & Blanz, 2001). Psy- 
chophysically, visual adaptation refers to the change in sensitivity (reduced respon- 
siveness caused by neural fatigue) of a neural population after prolonged exposure to 
the preferred visual pattern (adaptor) that results in a subsequent alteration of the 
appearance of a related visual stimulus (test), called adaptation aftereffect (AE). 

Visual adaptation has been considered as “the psychologist’s microelectrode” 
(Frisby, 1979), for it is a specific probe for the activity of different neural popula- 
tions. Applied to the study of the neural correlates of visual awareness in general and 
binocular rivalry suppression, the idea is that if visual adaptation to a certain pattern 
can survive perceptual suppression of the adaptor stimulus, then suppression occurs 
at a farther stage of visual processing compared to that underlying the analysis of 
that particular pattern, a stage of neural processing that is therefore not directly in- 
volved in visual consciousness (Crick & Koch, 1995, , 1998; Koch, 2007). 

The question is: to what extent can visual adaptation survive binocular rivalry 
suppression? Three are the possible outcomes when adapting to a visual stimulus 
rendered invisible by binocular rivalry suppression: full-strength adaptation, re- 
duced-strength adaptation, no adaptation. Finding a reduced-strength adaptation 
when the adaptor is presented during suppression suggests that at that stage of neu- 
ral processing binocular rivalry suppression is already operating attenuating, but not 
yet disrupting, neural responses to the suppressed image. When suppression is com- 
plete, no adaptation to the invisible stimulus would be observed, neural response to 
it being completely disrupted. 

Suppression has been found to have no influence at all on the strength of some 
basic adaptation aftereffects, indicating that they can arise also in the absence of the 
adapting stimulus from visual awareness: the tilt aftereffect (adaptation to tilted lines 
resulting in briefly perceiving vertical lines as tilted in the opposite direction, Gibson 
& Radner, 1937), the spatial frequency aftereffect (adaptation to evenly spaced lines 
of a certain spatial frequency producing either a contrast threshold elevation, 
Blakemore & Campbell, 1969) or a perceptual shift in spatial frequency of the test 
stimulus presented immediately after adaptation, Blakemore & Sutton, 1969), the 
orientation-contingent colour aftereffect (extended viewing of pars of coloured grat- 
ings results in perceiving non-coloured gratings as tinted with the opposite colours, 
McCollough, 1965) and the translational motion aftereffect (prolonged exposure to 


25 


Early cross-modal interactions and adult human visual cortical plasticity 


contours linearly moving in one direction causing perception of static targets as 
moving in the opposite direction, Mather, Verstraten, & Anstis, 1998) are not affect- 
ed by binocular rivalry suppression and show a strength that is comparable with that 
obtained when the adapting stimulus is presented monocularly or during dominance 
phases of binocular rivalry (Blake & Fox, 1974a, 1974b; Blake, Tadin, Sobel, Raissian, 
& Chong, 2006; O'Shea & Crassini, 1981a; Wade & Wenderoth, 1978). Consistently, 
motion priming (presenting a motion cue immediately before the presentation of 
bistable motion favours perception of the cued direction, Anstis & Ramachandran, 
1987) has also be demonstrated to survive binocular rivalry suppression (Blake, Ahl- 
strom, & Alais, 1999), being effective even during periods of temporary invisibility. 
Table 1.4.1. summarizes the outcomes of studies on adaptation aftereffects during 
binocular rivalry suppression. 


Level of unconscious processing during binocular rivalry (BR) and continuous flash suppression (CFS) as measured by strength of adaptation aftereffect (AE). 


Type of adaptation Adaptor Level of processing 
Afterimages (Als) Gray gratings during BR Reduced strength (Gilroy and Blake, 2005) 
Color gratings during CFS Reduced strength: the more completely the adaptor was suppressed, the more 
strongly the Al intensity was reduced (Tsuchiya and Koch, 2005) 
Tilt after-effects (TAEs) Gratings inclined 10° or 15° from the Full strength (Wade and Wenderoth, 1978) 
vertical during BR 
Spatial frequency aftereffects Square-wave gratings during BR Full strength: magnitude as measured by contrast threshold elevation and spatial 


frequency shift was determined solely by stimulus duration and independent of 
awareness (Blake and Fox, 1974a,b) 
Sinusoidal wave gratings during BR Full strength for high-contrast adaptors, reduced strength for low-contrast 


adaptors, as measured by contrast threshold elevation (Blake et al., 2006) 
Color aftereffects (CAEs) Color gratings during BR Full strength (White et al, 1978) 
Motion aftereffects (MAEs) Translational motion during BR Full strength (Wade and Wenderoth, 1978); interocular transfer of the MAE is not 


reduced by awareness (O'Shea and Crassini, 1981); full strength for high-contrast 
adaptors, reduced strength for low-contrast adaptors (Blake et al., 2006) 
Rotating spiral during BR Reduced strength: magnitude was proportional to the total duration of spiral 
visibility during adaptation (Wiesenfelder and Blake, 1990); 
Reduced (van der Zwan et al., 1993) 


Drifting plaid during BR 


Table 1.4.1. Adaptation Aftereffects (AE) measured during binocular rivalry suppression. 
Adapted from (Crick & Koch, 1998; Lin & He, 2009). 


Other types of adaptation aftereffects are only partially disrupted by binocular 
rivalry suppression and show a reduced strength compared to monocular condi- 
tions: negative afterimages (the illusory image appearing after exposure to a bright 
image, Craik, 1940) and complex forms of the motion after effect as spiral motion, 
drifting plaid and optic flow aftereffects (Mather, Verstraten, & Anstis, 1998) show 
reduced effect when the adaptor is suppressed during rivalry (Blake, 1997; Gilroy & 
Blake, 2005; Tsuchiya & Koch, 2005; van der Zwan, Wenderoth, & Alais, 1993). 

One difficulty arising when testing adaptation during suppression is that because 
of the stochastic nature of perceptual alternation during binocular rivalry, the dura- 
tion of the current suppressed phase is unpredictable, leading to non-homogeneous 
adaptation durations and to possible overlap of adaptation with dominance. These 
two methodological confounds could possibly underpin the survival of adaptation 
aftereffects found during perceptual suppression. In line with this idea, Blake et al 
(2006) have demonstrated that the strength of the preserved aftereffects depends on 
the proportion of time during which the adaptor had been effectively removed from 
awareness (Blake, Tadin, Sobel, Raissian, & Chong, 2006). They found reduced ad- 
aptation threshold elevation and motion aftereffects when effective suppression du- 
ration was controlled. These authors also demonstrated that the strength of the 
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spared adaptation aftereffect depended on contrast of the adapting stimulus, with 
lower contrast adaptors effect being more sensitive to binocular rivalry suppression. 

To compensate for the uneven, random duration of binocular rivalry suppres- 
sion phases, in recent studies the method of continuous flash suppression (CFS) has 
also been used, for it allows deep and constant suppression of a salient visual stimu- 
lus over extended periods of time (Tsuchiya & Koch, 2005). When one eye is contin- 
uously flashed with different, contour-rich, high-contrast random patterns (white 
noise, Mondrian patterns, scrambled images) at about 10 Hz (i.e. 100 ms per pat- 
tern), information presented to the other eye is perceptually suppressed for extended 
periods of time (up to 3 minutes or more, Figure 1.4.1.). Suppression provoked by 
the continuous flashes has been shown to summate, resulting not only in longer 
suppression periods, but also in deeper suppression of the other eye: detection 
thresholds of probes presented to the suppressed eye during CFS are in fact elevated 
of a 20-fold factor compared to monocular viewing, compared with a 3-fold eleva- 
tion observed during binocular rivalry (Tsuchiya, Koch, Gilroy, & Blake, 2006). 

Interestingly, the adaptation aftereffects surviving binocular rivalry suppression 
or being attenuated and not abolished by suppression originate from neural altera- 
tion in early visual cortices (V1-V2) entirely (Blakemore, Carpenter, & Georgeson, 
1970; Carpenter & Blakemore, 1973; Humphrey & Goodale, 1998; Maffei & Fioren- 
tini, 1973; Morrone, Burr, & Maffei, 1982; Shimojo, Kamitani, & Nishida, 2001) or in 
part (Huk, Ress, & Heeger, 2001; Movshon & Lennie, 1979), suggesting that binocu- 
lar rivalry suppression starts to be effective at the level of primary visual cortex, but it 
is not complete at that stage of visual processing. 


Stationary image Random flashed patterns 
in one eye in the other eye Typical percept 


Flash 


evel 


Figure 1.4.1. Continuous Flash suppression. Adapted from (Tsuchiya & Koch, 2005) 
Continuously flashed Mondrian patterns (10 Hz) in one eye provoke deep and stable percep- 
tual suppression of salient images presented to the other eye for a prolonged period of time 
(up to 3 minutes or more). 


In general, simple, basic visual stimuli that elicit responses early in the visual sys- 
tem are only partially affected by binocular rivalry suppression that instead com- 
pletely conceals responses to complex visual stimuli elaborated by higher order visu- 
al areas. In line with this idea, the rotating optic flow motion aftereffect (Wiesenfeld- 
er & Blake, 1990) as well as the paid induced motion aftereffect (van der Zwan, 
Wenderoth, & Alais, 1993), the face identity aftereffect (Moradi, Koch, & Shimojo, 
2005) and the illusory contour aftereffect (van der Zwan & Wenderoth, 1994) are 
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totally abolished when the adaptor is presented during periods of invisibility during 
binocular rivalry, these visual stimuli need therefore a conscious processing in order 
to be effective (Lin & He, 2009). 

The same trend is shown by visual priming experiments engaging complex 
stimuli: picture naming priming (advantage in reaction times and accuracy when 
naming visual pictures that had been previously presented even accidentally 
(Tulving & Schacter, 1990)) and word priming (fast recognition of scrambled words 
with prior exposure to words belonging to the same category, Meyer & 
Schvaneveldt, 1971) are both abolished if the priming stimulus is presented during 
binocular rivalry suppression (Cave, Blake, & McNamara, 1998; Zimba & Blake, 
1983). Coherently with this line of evidence, ERP responses (N400) to meaningful 
visual stimuli have been found to be silenced by binocular rivalry suppression (Kang, 
Blake, & Woodman, 2011). The complex visual stimuli listed above are elaborated by 
visual areas at a subsequent stage compared to the early cortices in which neural ac- 
tivity is only reduced by suppression. This evidence supports a hierarchical organiza- 
tion of binocular rivalry, with suppression depth increasing with increasing com- 
plexity of the visual stimuli and along both the dorsal and the ventral visual pathway 
(Nguyen, Freeman, & Alais, 2003). 


1.4.2. Where does suppression take place in the visual brain? 


The attenuation of neural responses caused by binocular rivalry suppression has 
been demonstrated to initiate early in the visual system, with BOLD signal amplitude 
awareness-dependent fluctuations being measured already at the thalamic level in 
LGN (Haynes, Deichmann, & Rees, 2005; Wunderlich, Schneider, & Kastner, 2005) 
and in monocular regions of the primary visual cortex, as demonstrated by Tong 
and Engel using a clever experimental paradigm taking advantage of the blind-spot 
representation (Tong & Engel, 2001). Neural activity modulation reflecting percep- 
tual states of binocular rivalry has been further demonstrated at the level of the pri- 
mary visual cortex using neuroimaging techniques (Levi, 2005; Lumer, Friston, & 
Rees, 1998; Polonsky, Blake, Braun, & Heeger, 2000), indicating that binocular rival- 
ry suppression is already effective in V1, that is therefore involved in awareness. Fur- 
thermore, BOLD signal modulation in V1 has also been demonstrated to reflect the 
perceptual travelling waves experienced by the observer during transition periods 
following retinotopic maps (Lee, Blake, & Heeger, 2007). The role of the primary 
visual cortex in mediating visual awareness has been also confirmed by Haynes and 
Rees, showing that V1 activity during binocular rivalry when analyzed by a support- 
vector machine could predict the observer’s conscious state with accuracy approach- 
ing 100% (Haynes & Rees, 2005). 

The BOLD signal is a measure of the concentration of metabolic markers reflect- 
ing only indirectly neural activity modulation that also comprises neural events oth- 
er than action potentials (Logothetis, 2003), using other neuroimaging techniques 
that measure neural activity directly is useful for expanding and reinforcing the evi- 
dence of an involvement of early visual cortices in mediating perceptual alternations 
during binocular rivalry. Fluctuations of the signal strength coupled with binocular 
rivalry alternations have been revealed using EEG techniques (VER, visual evoked 
responses, Brown & Norcia, 1997; de Labra & Valle-Inclan, 2001; Roeber & Schrog- 
er, 2004) and MEG (magnetoencephalography, Srinivasan & Petrovic, 2006; Tononi, 
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Srinivasan, Russell, & Edelman, 1998), that provide a direct measure of cerebral ac- 
tivity but cannot precisely localize the neural locus where that activity is originated 
because of the low spatial resolution provided by the two techniques. 

Taken together, the psychophysical results showing attenuation, but not cancellation 
of adaptation aftereffects thought to reflect neuronal activity in primary visual cortex and 
the neuroimaging evidence listed above, demonstrate that binocular rivalry suppression 
inhibits V1 activity to a certain extent, but does not extinguish early visual responses, for 
basic visual information can be analyzed despite perceptual suppression. This is con- 
firmed by evidence showing that when suppressed by binocular rivalry, early visual fea- 
tures such as orientation (Pearson & Clifford, 2005a; Stuit, Paffen, van der Smagt, & Ver- 
straten, 2011) and colour (Hong & Shevell, 2009) of a visual stimulus can alter the ap- 
pearance of the dominant stimulus, escaping suppression mechanisms. 

At the first stages of visual analysis suppression is not complete, some information 
survives even if weakened compared to dominance. Ascending the hierarchy of visual 
processing, suppression is thought to increase in depth both along the ventral and the 
dorsal pathway (Goodale & Milner, 1992), as demonstrated by psychophysical evidence 
measuring detection thresholds to probes presented during suppression of visual stimuli 
of increasing complexity, that are therefore processed by visual areas at different levels 
(Alais & Melcher, 2007; Nguyen, Freeman, & Alais, 2003). 

Further evidence in favour of an increasing strength of suppression proceeding 
along the stream of visual analysis comes from electrophysiological recordings in awake 
monkeys experiencing binocular rivalry (Leopold & Logothetis, 1996; Sheinberg & Logo- 
thetis, 1997): the number of neurons showing reduced responses during suppression 
phases of binocular rivalry augments progressing along the visual pathway (Figure 
1.4.2.), with total response suppression being achieved in inferotemporal cortex (IT), that 
comprises an ensemble of object selective areas encoding complex objects in a view- and 
position-invariant way (Ungerleider, Galkin, & Mishkin, 1983). 

The disruption of neural activity caused by binocular rivalry suppression in the- 
se high-level visual areas is responsible for the failure of suppressed complex images 
in surviving binocular rivalry suppression, for the annihilation of IT responses re- 
sults in impaired object recognition and semantic processing, explaining the absence 
of picture naming and word priming (Cave, Blake, & McNamara, 1998; Zimba & 
Blake, 1983), as well as the lack of adaptation to complex objects like faces (Moradi, 
Koch, & Shimojo, 2005) observed during suppression. Similar results have been ob- 
tained measuring BOLD responses in high-level visual areas both of the dorsal 
(Moutoussis, Keliris, Kourtzi, & Logothetis, 2005) and of the ventral pathway 
(Moutoussis & Zeki, 2002; Tong, Nakayama, Vaughan, & Kanwisher, 1998). For ex- 
ample, Tong et al (1998) measured BOLD responses in the human fusiform face area 
(FFA) and the parahippocampal place area (PPA) during binocular rivalry between 
faces and house stimuli presented dichoptically (Tong, Nakayama, Vaughan, & 
Kanwisher, 1998). The FFA is a face-selective area that is activated by faces and not 
by other visual objects (Kanwisher, McDermott, & Chun, 1997), while the PPA is 
selectively activated by places but does not respond to faces (Epstein & Kanwisher, 
1998). Interestingly, Tong et al found that BOLD activity in those selective areas fol- 
lowed observer’s visual perception, fluctuating according to reported perceptual al- 
ternations. Moreover, the amplitude of the BOLD response measured during binoc- 
ular rivalry was comparable to that measured during non-rivalrous perception (Fig- 
ure 1.4.3.). This results demonstrate that in object-selective areas of the human brain 
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binocular rivalry is completely resolved, with neural activity reflecting only 
conscious perception (Tong, Nakayama, Vaughan, & Kanwisher, 1998). 


Frequency (%) 


0 

Figure 1.4.2. Distribution of perception-related activity in the monkey’s visual cortex. 
Adapted from (Logothetis, 1998). The number of neurons showing perception-modulated 
activity increases along the visual pathway and is maximum (91% of total neurons) in the IT 
cortex when monkeys are presented with binocular rivalry between complex visual stimuli. 
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Figure 1.4.3. Bold activity in FFA and PPA measured during binocular rivalry and during 
non-rivalrous stimulation. Adapted from (Tong, Nakayama, Vaughan, & Kanwisher, 1998). 
While lying in a magnetic resonance imaging (MRI) scanner, the observer is presented with 
dichoptic complex visual stimuli (house and face, panel A) generating binocular rivalry. The 
amplitude of the BOLD signal in PPA and FFA (high level extrastriate areas in the temporal 
cortex) was modulated following observer’s perception during rivalry and was almost identi- 
cal to that observed during rivalry replay (Panel B). 
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An exceptional insight into the neural mechanisms underlying binocular rivalry 
dynamics in the human ventral visual pathway has been obtained by Kreiman, Fried 
and Koch (2002), who measured in vivo single neurons responses in the medial 
temporal lobe (MLT) of human observers implanted with depth electrodes to local- 
ize the focus of seizure onset in pharmacologically untreatable epilepsy (Kreiman, 
Fried, & Koch, 2002). When binocular rivalry (in the form of flash suppression, 
Blake, Westendorf, & Fox, 1990; Wolfe, 1984) between the stimulus preferred by the 
cell and a random pattern was presented, the neuron responded during dominance 
of the preferred image (e.g. Bill Clinton’s face) and was silenced during its suppres- 
sion (Figure 1.4.4.), showing that MTL neurons activity followed visual perception 
during rivalry, reflecting the observer’s conscious perception. 

A large amount of evidence has been cumulating showing that binocular rivalry 
suppression totally abolishes neural activity along the ventral (vision-for-perception) 
visual pathway, but the same does not hold for the dorsal (vision-for-action) pathway 
(Goodale & Milner, 1992). Psychophysically, the fate of suppressed visual signals 
processed by the dorsal pathway has been demonstrated to be less dramatic than that 
of visual stimuli processed along the ventral pathway (e.g. faces, complex objects, 
meaningful scenes). For example, the suppressed direction of motion has been found 
to alter the dominant direction of motion (complex motion is processed by the dor- 
sal area V5-MT and the MT+ complex, Born & Bradley, 2005) when small visual 
stimuli are tested (Andrews & Blakemore, 1999) and priming to manipulable objects 
(processed by the dorsal areas V3A and V7 and IPS, Grill-Spector, Kushnir, Edel- 
man, Itzchak, & Malach, 1998; Grill-Spector, Kushnir, Hendler, & Malach, 2000) has 
been shown to survive binocular rivalry suppression in a categorization task (Al- 
meida, Mahon, Nakayama, & Caramazza, 2008). Similarly a residual BOLD activa- 
tion has been found in response to images of manipulable tools rendered perceptual- 
ly invisible by CFS (Fang & He, 2005). Finally, suppression of rivalring visual stimuli 
can be shortened when the observer voluntary controls the direction of motion of 
the suppressed visual stimulus (Maruya, Yang, & Blake, 2007). That some trace of 
perceptually suppressed image can be retained by the dorsal pathway agrees with the 
model proposed by Goodale and Milner (1992), according to which the dorsal path- 
way encodes visual information useful for action and not for object recognition, and 
is therefore not necessarily linked to visual awareness (Goodale & Milner, 1992). 

Faces are a special category of visual stimuli, with a dedicated visual area (FFA, 
Kanwisher, McDermott, & Chun, 1997). Even though binocular rivalry suppression 
destroys neural activity in the FFA (Tong, Nakayama, Vaughan, & Kanwisher, 1998), 
some information conveyed by facial expression can survive suppression. Visual in- 
formation charged with a negative emotion is thought to be processed by the visual 
system in an compulsory, non-conscious way, automatically capturing attention taking 
advantage of priority processing, for rapidly detecting threats is essential for survival. 
For example, angry, threatening faces have been shown to have a perceptual advantage 
over happy or neutral faces in visual search, known as “anger superiority” (Hansen & 
Hansen, 1988). In line with this evidence, BOLD activity has been found in the left 
amygdala and in the STS in response to fearful faces rendered invisible by binocular 
rivalry suppression (Jiang & He, 2006; Pasley, Mayes, & Schultz, 2004; Williams, Mor- 
ris, McGlone, Abbott, & Mattingley, 2004), while no response to neutral faces was rec- 
orded. Psychophysically, images of fearful faces have been found to be suppressed for a 
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shorter time compared to happy and neutral faces during CFS (Stein & Sterzer, 2012; 
Sterzer, Hilgenfeldt, Freudenberg, Bermpohl, & Adli, 2011; Tsuchiya, Moradi, Felsen, 
Yamazaki, & Adolphs, 2009; Williams, Morris, McGlone, Abbott, & Mattingley, 2004; 
Yang, Zald, & Blake, 2007). Consistently with the idea of a privileged conscious access 
for emotional information, an initial bias in favour of threatening faces has been 
demonstrated to be stronger in individual with an anxiety disorder (Singer, Eapen, 
Grillon, Ungerleider, & Hendler, 2012). Finally, emotional arousing images (e.g. nudi- 
ty) have been shown to escape perceptual suppression caused by CFS (Jiang, Costello, 
Fang, Huang, & He, 2006). 
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Figure 1.4.4. Single cell response to the preferred stimulus during binocular rivalry in human 
medial temporal lobe (MLT). Adapted from (Kreiman, Fried, & Koch, 2002). 


In general, investigating the fate of suppressed visual images is a unique tool to 
understand the neural mechanisms underlying conscious visual perception (Crick & 
Koch, 1998; Lin & He, 2009). To summarize, binocular rivalry suppression is already 
effective at early stages of visual analysis (LGN, primary visual cortex), where the 
signal associated with the suppressed image is reduced in strength, but not de- 
stroyed. Information processed by these early visual areas, though weakened, sur- 
vives suppression and can be used outside of visual awareness, as demonstrated by 
adaptation and priming experiments as well as by experiment showing that the sup- 
pressed image can alter the appearance of the dominant one. Ascending the visual 
hierarchy, suppression increases in depth, especially along the ventral pathway, 
where suppressed visual information about complex objects and meaningful images 
is disrupted. Complex stimuli can escape binocular rivalry suppression when they 
provide a behavioural advantage, either for action (e.g. manipulable tools) or for re- 
action to threat (e.g. fearful faces), gaining a priority access to consciousness. 


1.5. Binocular Rivalry and Attention. 


Binocular rivalry and selective attention share similar characteristics and an 
overlapping neural network, that is why the two phenomena have been proposed to 
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be strictly linked. Both selective attention and rivalry, in fact, involve a perceptual 
choice among competing visual stimuli, with a winner-takes-it-all outcome in which 
the selected image dominates over the non-selected one (Leopold & Logothetis, 
1999). Furthermore, a distributed fronto-parieto-occipital network of cerebral areas 
has been shown to be crucial both for the exertion of top-down attentional selection 
(Corbetta & Shulman, 2002; Desimone & Duncan, 1995; Kastner & Ungerleider, 
2000; Posner & Dehaene, 1994) and for binocular rivalry (Cosmelli et al., 2004; 
Knapen, Brascamp, Pearson, van Ee, & Blake, 2011; Lumer, Friston, & Rees, 1998; 
Miller et al., 2000; Srinivasan, Russell, Edelman, & Tononi, 1999; Sterzer & Rees, 
2008). Figure 1.5.1. shows the neural network mediating selective attention (panel A) 
and rivalry (panel B), note that in the case of binocular rivalry it is not clear whether 
the stronger activity found in the fronto-parietal network for binocular rivalry vs 
pseudo-rivalry display reflects a feedback or a feedforward signal with early visual 
areas involved in processing binocular rivalry. 

The role of attention on the dynamics of binocular rivalry has been attracting the 
interest of visual scientists since the discovery of binocular rivalry itself. When de- 
scribing tor the first time binocular rivalry in 1838, Wheatstone made the observa- 
tion that “it does not appear to be in the power of the will to determine the appear- 
ance of either of the two letters” (see paragraph 1.1.1. for the extended quote of 
Wheatstone, 1838), raising therefore the question whether binocular rivalry could be 
under the control of voluntary attention. Unlike Wheatstone, almost one century 
later, Herman von Helmoltz (1925) considered binocular rivalry as a “psychic act”, 
for he claimed that he could voluntarily control the dynamics of binocular rivalry 
selecting one or the other image according to his will. Fom this result Helmoltz con- 
cluded that binocular rivalry was a high-level phenomenon (Helmoltz, 1925). How- 
ever, the early studies on attentional control over binocular rivalry did not control 
for eye movements, that have been shown to interact with binocular rivalry both by 
influencing dominance of one image over the other and by influencing switching 
rate (Kalisvaart, Rampersad, & Goossens, 2011; Peckham, 1936; van Dam & van Ee, 
2006). 

The first systematic study on the role of attention on the dynamics of binocular 
rivalry dates back to 1978, when Lack reviewed the literature and performed careful- 
ly designed experiments in which he demonstrated that perceptual alternations dur- 
ing binocular rivalry could be modulated by selective attention even after that pe- 
ripheral factors like eye-movements were ruled out, and that the amount of atten- 
tional influence on binocular rivalry increased with practice (Lack, 1978). Interest- 
ingly, the same conclusion was drawn almost 30 years later by Van Dam and Van Ee 
(2006), that re-considered the role of saccadic eye-movements in mediating atten- 
tional control over rivalry. These authors found a correlation between the exertion 
of saccades and the occurrence of perceptual switches, so that observers could use 
eye-movements to trigger an alternation, but they failed in finding a difference in eye 
movements rate between normal viewing conditions and conditions in which ob- 
servers were asked to attentionally modulate rivalrous perception, van Dam & van 
Ee, 2006). Even though Lack clearly showed that attention could play a role in bias- 
ing binocular rivalry, Helmoltz’ belief that binocular rivalry depended on attentional 
selection and was under complete attentional control (Helmoltz, 1925) has been re- 
examined, the role of attention being scaled down to modulation rather than control 
over rivalry. 
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Figure 1.5.1. Fronto-parietal neural networks involved in selective attention (A) and binocu- 
lar rivalry (B). (A) adapted from (Corbetta & Shulman, 2002) and (B) adapted from (Lumer, 
Friston, & Rees, 1998)) 


In the next paragraphs we will review recent studies investigating the role of at- 
tention on binocular rivalry separately for voluntary/endogenous attention and in- 
voluntary/exogenous attention. Voluntary attention refers to an active selection of a 
visual stimulus by the observer, reflecting a top-down process, while exogenous at- 
tention refers to an automatic selection in which a particular feature of a visual stim- 
ulus captures observer’s attention and reflects therefore a bottom-up process (Bun- 
desen & Habekost, 2008; Wright & Frenzel, 2008). 


1.5.1. Voluntary/Endogenous attention and binocular rivalry 


After Lack’s observations (Lack, 1978), research on the modulatory effects of 
binocular rivalry has been taken up by Ooi and He (1999), who exploited the Chesh- 
ire Cat effect (Duensing & Miller, 1979) that occurs when one eye is presented with 
an image, while a blank is presented to the other eye. Under these conditions, the 
image dominates over the blank, introducing a perturbation in the visual field of the 
eye that sees the blank leads the image presented to the other eye to be suppressed. 
Ooi and He (1999) found that driving voluntary attention to a rivalring stimulus 
counteracted the tendency of the Cheshire Cat effect to suppress the image, volun- 
tary attention retaining the selected visual stimulus dominant for longer periods 
(Ooi & He, 1999). 

A similar result was found in a study performed by Meng & Tong, who con- 
fronted the role of attention in mediating bistable perception and binocular rivalry 
(Meng & Tong, 2004). When the authors instructed participants to voluntarily hold 
one of the rivalring images at the expenses of the other, they found that attentive 
modulation of rivalry was possible (Figure 1.5.2.), but that observers could only 
slightly lengthen dominance of the attended visual stimulus (+13% on average), 
whereas, attentional control over another bistable visual stimulus (Necker cube) was 


34 


Claudia Lunghi 


stronger (+37% on average). One year later, van Ee, van Dam and Brouwer con- 
firmed the resistance of binocular rivalry to voluntary attention compared to other 
types of bistable stimuli (van Ee, van Dam, & Brouwer, 2005). 

From this result, Meng & Tong concluded that binocular rivalry was a more au- 
tomatic, stimulus-driven process compared to bistable perception, being probably 
mediated by lower-level neural events (Meng & Tong, 2004). Coherently with this 
hypothesis, in another experiment Meng & Tong found that observers’ voluntary 
modulation of overall switching rate was as strong for binocular rivalry than for the 
Necker cube, showing that alternation rate modulation and selection of one image 
are distinct processes, the first being mediated by lower-level mechanisms (switching 
rate is in fact influenced by low level, physiological factors like, for example eye 
blinks, Peckham, 1936, and can be changed by modulating stimulus strength, see 
paragraph 1.2.3 , with stronger stimuli alternating faster, Levelt, 1966). 

The distinction between selection of a particular visual stimulus and modulation 
of switching rate has been confirmed by Paffen, Alais and Verstraten (2006), who 
demonstrated that alternation rate during binocular rivalry was slower when atten- 
tion was diverted from the rivalring stimuli (orthogonal gratings) by a concurrent 
visual task (surround motion direction discrimination). The slowdown of the 
switching rate also correlated with the difficulty of the concurrent task and was simi- 
lar to the effect of reducing contrast of the rivalring images (Levelt, 1966), from this 
the authors concluded that attention acts on binocular rivalry by actually boosting 
the apparent contrast of the visual stimuli, speeding therefore the alternation rate 


(Paffen, Alais, & Verstraten, 2006). 
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Figure 1.5.2. Influence of voluntary attention on the dynamics of bistable perception (Necker 
cube) and binocular rivalry. Adapted from (Meng & Tong, 2004) 

When asked to attend to one of the two bistable visual stimuli by holding it in dominance, 
observer’s attentional control was greater on the Necker cube (Panel A) than on binocular 
rivalry (Panel B). 


Several other studies have proposed that attention interacts with binocular rival- 
ry by increasing effective contrast of the visual stimuli (Chong & Blake, 2006; Chong, 
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Tadin, & Blake, 2005; Paffen, Alais, & Verstraten, 2006; Paffen, Hooge, Benjamins, & 
Hogendoorn, 2011), for attending to a visual stimulus has been shown to increase its 
apparent contrast (Carrasco, Ling, & Read, 2004) and also to affect neural responses 
in a similar way to that of contrast increasing (Reynolds & Chelazzi, 2004). This in- 
terpretation is still debated, because the effect of attention seems to violate Levelt’s 
second proposition that increasing contrast in one eye shortens durations of the 
stimulus presented to the other eye and therefore diminish suppression of the 
stronger visual stimulus (Levelt, 1965). For example, Chong, Tadin and Blake (2005) 
have shown that performing an attentional task on one of the rivalring stimuli (i.e. 
monitoring small changes in spatial frequency) increases dominance duration of the 
attended stimulus up to +50% compared to baseline, but does not affect the other 
stimulus duration (Chong, Tadin, & Blake, 2005). 

This experiment shows that attentional modulation of binocular rivalry can be 
enhanced increasing behavioural relevance of the visual stimuli and task (+50% 
modulation compared to +13% found by Meng and Tong, 2004), but argues against 
the “increasing apparent contrast” interpretation of the effect of attention on bin- 
ocular rivalry, for it clearly violates Levelt’s second proposition (Levelt, 1965). An 
accepted explanation of this violation is that one cannot voluntarily attend to an un- 
seen visual stimulus and therefore voluntarily attention cannot interact with a sup- 
pression phase during rivalry. In this light, the result obtained by Hancock and An- 
drews (2007) is controversial, for it is the only evidence showing an effect of volun- 
tary attention both on dominance and on suppression of the attended stimulus 
(Hancock & Andrews, 2007). 
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Figure 1.5.3. Attentional gain on binocular rivalry provoked by a congruent sound. Adapted 
from (van Ee, van Boxtel, Parker, & Alais, 2009) 

Attentional modulation on binocular rivalry between a looming visual stimulus and a rotat- 
ing radiant grating is enhanced by the concomitant presentation of a congruent auditory 
(Panel A) or tactile stimulus. The auditory signal does not bias binocular rivalry dynamics on 
its own (Experiment 4), but is effective in boosting attentional modulation on the rivalring 
congruent percept. Increasing behavioural relevance increases attentional gain, that is maxi- 
mum for trimodal stimulation (audition + touch, Panel B). 


The importance of behavioural relevance in enhancing effectiveness of attention 
in influencing the dynamics of binocular rivalry has been confirmed by a recent 
study (van Ee, van Boxtel, Parker, & Alais, 2009) showing that attentional control 
over one of the rivalring visual stimuli can be enhanced if a congruent signal is pre- 
sented in another sensory modality (audition). The authors also demonstrated that, 
in order to observe the attentional gain, the auditory signal needed to be perfectly 
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synchronized with the looming frequency of the attended visual signal (Figure 
1.5.3.A) and that increasing behavioural relevance by introducing another congruent 
signal (touch) further increased the attentional gain (Figure 1.5.3.B). Interestingly, 
the congruent auditory stimulus enhanced attentional modulation of binocular ri- 
valry only by prolonging dominance of the attended visual stimulus and did not af- 
fect its suppression, a result that agrees with the majority of the literature (Chong, 
Tadin, & Blake, 2005; Meng & Tong, 2004; Paffen & Alais, 2012; van Ee, Noest, Bras- 
camp, & van den Berg, 2006; van Ee, van Dam, & Brouwer, 2005). 


1.5.2. Involuntary/Exogenous attention and binocular rivalry 


Exogenous attention reflects a bottom-up modulation of neural activity trig- 
gered by a salient visual stimulus that automatically attracts attention (Bundesen & 
Habekost, 2008; Wright & Frenzel, 2008). Endogenous and exogenous attention are 
two distinct processes that are thought to be elaborated by different neural networks 
(Corbetta, Kincade, & Shulman, 2002; Posner & Dehaene, 1994), it is therefore plau- 
sible that they have different effects on the dynamics of binocular rivalry, that is a 
complex phenomenon mediated by a distributed neural network (Alais & Blake, 
2005; Blake & Logothetis, 2002; Tong, Meng, & Blake, 2006). 

As discussed in the previous paragraph, voluntary attention influences binocular 
rivalry mainly by prolonging dominance duration of the attended image. In 1999 
Ooi and He investigated whether, differently from voluntary attention, involuntary 
attention could revert a suppressed image to dominance. It is well known that a 
transient change presented to the suppressed image can produce a perceptual switch 
(Walker & Powell, 1979), taking advantage of this phenomenon, Ooi and He intro- 
duced a pop-out element in the suppressed image, that attracted automatically exog- 
enous attention and found that involuntary attention attracted to a cued location 
had the effect of bringing back the suppressed image to dominance (Ooi & He, 
1999). Similarly, Paffen and Von der Stigchel (2010) found that a pop out cue to a 
particular location of the visual field increased the occurrence of perceptual switches 
to the rivalring stimuli presented in the cued locations and concluded that exoge- 
nous attention is able to initiate a perceptual alternation rescuing the suppressed 
visual signal (Paffen & Van der Stigchel, 2010). 

Object-based exogenous attention has also been found to strongly bias the onset 
of rivalry, that, as discussed in paragraph 1.3.3. is thought to be a distinct, though 
related process compared to sustained rivalry (Stanley, Forte, Cavanagh, & Carter, 
2012). For example, if prior to the onset of rivalry attention is exogenously cued by 
briefly translational movement of one of two overimposed transparent rotating tex- 
tures (Figure 1.5.4.), onset rivalry will be biased in favor of the cued surface (Mitch- 
ell, Stoner, & Reynolds, 2004). A similar result has then been obtained by Chong 
and Blake (2006) using as visual stimuli orthogonal gratings and as exogenous cue 
an increment in contrast or a little orientation change (Chong & Blake, 2006), and 
by Hancock and Andrews (Hancock & Andrews, 2007). Chong and Blake also re- 
ported that onset rivalry could be biased by voluntarily orienting attention to one of 
the rivalring stimuli before the dichoptic presentation (Chong & Blake, 2006). 
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Figure 1.5.4. Influence of object-based attention on onset rivalry. Adapted from (J. F. Mitch- 
ell, Stoner, & Reynolds, 2004) 

The onset of rivalry between dichoptic surfaces rotating in opposite directions is be systemat- 
ically bias by exogenously attracting object-based attention to one of the rivalring stimuli 
(brief translation of one of the transparent surfaces) before the instigation of rivalry (Panel 
A). The first coherent percept, following a brief period of fusion, is most of the time in favour 
of the cued surface (Panel B). 


1.5.3. Is attention necessary for binocular rivalry? 


What happens if attention is diverted from the rivalring images? Would alterna- 
tions occur anyways? Several studies have addressed this issue, starting from the in- 
fluential paper by Ooi and He (1999) mentioned above: in the last experiment the 
authors showed that the onset of rivalry was severely affected if attention was divid- 
ed in a dual-task paradigm (concomitant Vernier task), but even if alternations un- 
der divided-attention condition occurred less frequently, binocular rivalry was not 
abolished (Ooi & He, 1999). A similar conclusion was drawn by Paffen, Alais and 
Verstraten (2006), that, testing sustained alternations during prolonged rivalry 
showed that (as discussed above) performing a motion detection task did not stop 
perceptual alternations, bus slowed them down (Paffen, Alais, & Verstraten, 2006). 
Moreover, by increasing the number of rivalring images across the visual field, dis- 
tributing therefore spatial attention to several targets, Paffen and Hooge (2011) also 
reported a reduction of alternation rate, confirming that dividing attention during 
binocular rivalry slows alternations but does not abolish them (Paffen, Hooge, Ben- 
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jamins, & Hogendoorn, 2011). Interestingly, the same result is observed when atten- 
tion is cross-modally diverted from the rivalring images: attending to an auditory 
stimulus in fact has been found to slow alternation rate in a similar way compared to 
intra-modal attentional division (Alais, van Boxtel, Parker, & van Ee, 2010). 

All the studies mentioned above performed a dual task in which attention was 
divided between reporting binocular rivalry alternations and a concurrent task, so, 
even if attention was diverted from the rivalring visual stimuli, observers still had to 
track alternations, allocating therefore some attentional resources to the rivalring 
display. Recently, two clever studies have demonstrated that binocular rivalry is 
abolished if attention is totally diverted from the visual stimuli (i.e. observers are not 
required to track dominance and only attend to the concurrent task). Using the fre- 
quency-tagging SSVEP technique proposed by Brown and Norcia (Brown & Norcia, 
1997) that allows to precisely track perceptual alternations from the EEG signal 
(Brown & Norcia, 1997), Zhang et al (2011) showed that when attention was divert- 
ed from the rivalring stimuli, alternations stopped and the EEG signal was similar to 
that obtained with a combination of the two stimuli, strongly pointing to the elimi- 
nation of rivalry provoked by removal of attention (Zhang, Jamison, Engel, He, & 
He, 2011). 

That attention is fundamental for binocular rivalry has been recently confirmed 
by psychophysical evidence: Brascamp and Blake (2012) invented a new experi- 
mental paradigm in which observers’ rivalrous perception could be predicted, so that 
during the attentional task concomitant to binocular rivalry, rivalrous alternations 
tracking was not necessary. To increase the degree of rivalrous perception predicta- 
bility, the authors took advantage of the phenomenon of “flash suppression” (Wolfe, 
1984), in which the onset of rivalry can be controlled by the prior presentation of 
one of the two rivalring images, leaving unaltered the subsequent alternation pro- 
cess. Brascamp and Blake (2012) confronted conditions in which visual perception 
during binocular rivalry was reported by the observers (attended rivalry) and condi- 
tions in which attention was diverted to a RSVP task and fusable grating were pre- 
sented (absent rivalry) with conditions where attention was diverted to the RSVP 
task (unattended rivalry), but rivalring stimuli were presented. 

The experimental paradigm is illustrated in Figure 1.5.5.A: every trial was com- 
posed of three phases, the first phase was identical for all conditions and consisted of 
2 seconds monocular presentation of one of the rivalring stimuli that will produce 
the phenomenon of flash suppression in which at the onset of rivalry the other visual 
stimulus will be perceived most of the times. The second phase varied between con- 
ditions, in the attended and unattended condition the rivalring gratings and a se- 
quence of letters for a RSVP task were presented simultaneously for 2 seconds, in the 
attended condition observers were asked to ignore the letters and report their rival- 
rous visual perception while in the unattended condition they were asked to ignore 
the rivalring gratings and perform the RSVP task. The absent condition was similar 
to the unattended one, but instead of orthogonal gratings fusable gratings were pre- 
sented. The third phase was identical for all conditions: only the rivalring grating 
were presented and observers were asked to report which one dominated. 

Results are shown in Figure 1.5.5.B In the attended condition, as expected, dur- 
ing phase two visual perception was biased towards the visual stimulus forced by 
flash suppression, during phase three an echo of flash suppression was observed with 
a rebound in favour of the forced stimulus after a period of dominance of the other 
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one. This result can be considered as a hallmark of rivalry. In the absent condition, 
during phase three no echo of flash suppression was observed and the trace was flat 
at chance level, this result constitutes instead the hallmark of no-rivalry. Interesting- 
ly, visual perception after the attentional task in the unattended rivalry condition 
did not show the echo of flash suppression and was comparable to the absent rivalry 
condition in which during the RSVP task visual stimuli were fusable gratings (Figure 
1.5.5.B). From this result the authors concluded that binocular rivalry does not oc- 
cur in the absence of attention. 


That binocular rivalry does not occur in the absence of attention is a controver- 
sial result, for it disclaims the evidence that binocular rivalry is mediated by adapta- 
tion and early reciprocal inhibition between monocular representations of the visual 
stimuli (Alais, Cass, O'Shea, & Blake, 2010), that is supposed to be a low-level, auto- 
matic process and needs further investigation. The experimental evidence reviewed 
above nevertheless indicate that selective attention and bistable perception are strict- 
ly linked and that attention can modulate visual rivalrous perception, when dealing 
with a binocular rivalry experiment the role of attention therefore needs to be taken 
seriously into account. 
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Figure 1.5.5. inattention abolishes binocular rivalry, psychophysical evidence. Adapted from 
(Brascamp & Blake, 2012). 

Panel A shows a diagram of the different experimental conditions. In Panel B the average 
proportion of times in which the visual stimulus forced by flash suppression was perceived is 
reported as a function of time from the onset of phase two for the attended condition and 
from the onset of phase three for the other two conditions. In the unattended condition, the 
average proportion of dominance was flat at chance, with no echoes of flash suppression. 
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Chapter 2 
Cross-modal interactions during binocular rivalry 


2.1. Introduction. Multisensory Processing. 


Robust perception of the external world is achieved by combining information 
arising from different sensory modalities (including vision, audition and touch), for 
none of them alone can provide information that is reliable enough for all the differ- 
ent properties of the external world. Integrating multisensory information is there- 
fore a strategy that improves the quality of perception, allowing efficient interaction 
with the environment in all circumstances (Ernst & Bulthoff, 2004). In this light, it is 
surprising that until recently, sensory research focused mainly on the single sensory 
modalities independently. The blooming of research on multisensory processing has 
in fact occurred during the last decade, challenging the traditional principles that 
held before. The classic approach to multisensory processing relied on the assump- 
tion that sensory information was first analyzed by the unisensory areas, and then 
combined at a later stage (that was considered the “binding problem”, Treisman & 
Gelade, 1980; von der Malsburg, 1999), in higher level cerebral areas that were there- 
fore named “polysensory areas” (Benevento, Fallon, Davis, & Rezak, 1977; Felleman 
& Van Essen, 1991; Jones & Powell, 1970). In the next paragraphs we will review 
some of the most important studies that have recast multisensory research in a dif- 
ferent light. 


2.1.1. Multisensory processing in the superior colliculus (SC). 


The recent explosion of research on multisensory processing is due to the ad- 
vance in comprehension of the neural mechanisms underlying unisensory pro- 
cessing and to the pioneering work by Meredith and Stein (1983), who reported the 
existence of multisensory neurons (i.e. neurons responding to inputs from more 
than one sensory modality) in the deep layers of the superior colliculus (SC) of cats 
(Meredith & Stein, 1983). This finding pointed out the importance of multisensory 
combination to increase the probability of detecting and identifying objects in the 
external world. The superior colliculus in fact is a subcortical structure that plays a 
leading role in orienting behaviours such as directing the gaze to an object of inter- 
est. The receptive fields of multisensory collicular neurons encode the external space 
according to “spatiotopic coordinates”, resulting in functional maps of the external 
environment in which multiple receptive fields (one for each sensory modality) 
overlap and respond to a particular region of the external space (Meredith & Stein, 
1990). 
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In different experiments, Meredith and Stein showed that, in order to elicit a re- 
sponse of the multisensory neuron in the SC, the signals from the different sensory 
modalities need to be aligned both in space (Meredith & Stein, 1986, , 1996) and in 
time (Meredith, Nemitz, & Stein, 1987). These constraints ensure that multisensory 
integration occurs only between signals representing the same object. Interestingly, 
when spatio-temporally congruent stimuli are presented in different sensory modali- 
ties, the multisensory neurons of the SC show a peculiar non-linear behaviour in 
their response. For example, when the unisensory inputs are weak, the response of 
the multisensory neurons is greater than the sum of the individual signals (Meredith 
& Stein, 1996). This response enhancement behaviour is named “superadditivity” 
and has been considered as a physiological benchmark of multisensory integration. 
Superadditivity is functionally important, for it ensures the detection of otherwise 
weak stimuli that would not be detected on their own but could be behaviourally sa- 
lient. In line with this interpretation, superadditivity has been shown to be more 
likely to be found as the strength of the unisensory inputs decreases, a principle 
called “inverse effectiveness” (Stanford, Quessy, & Stein, 2005; B. E. Stein & Stan- 
ford, 2008). Both superadditivity and inverse-effectiveness are illustrated in figure 
2.1.1. (reported from Stein & Stanford, 2008). 


Figure 2.1.1. Non-linear characteristics of multisensory integration improve detection. 
Adapted from (B. E. Stein & Stanford, 2008). 

The orienting behaviour of a woman and a cat to the approach of a dog based on vision and au- 
dition: when the dog is far (1), the visual and auditory signals associated with the dog are weak, 
but response of the collicular multisensory neuron is stronger than the sum of the unisensory 
components (superadditivity). As the dog approaches, the unisensory signals become stronger 
and multisensory response decreases proportionally, becoming additive (2) and subaddittive (3) 
for vigorous unisensory signals. In this latter case the multisensory response is still stronger than 
the most vigourous unisensory signal, still improving detection of the dog. 
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As opposed to superadditivity, response depression of multisensory neurons has 
been observed for stimuli misaligned in space (Meredith & Stein, 1996), a mecha- 
nism that ihnhibits multisensory integration of objects occupying spatially separated 
positions. Taken together, the physiological properties of multisensory neurons 
guarantee fast and accurate orienting behaviours in response of congruent multisen- 
sory stimuli and slower and less accurate reactions for incongruent multisensory 
stimuli. Therefore, multisensory integration provides an advantage by improving the 
effectiveness of the interaction with the external world and, possibly, the survival of 
the organism, by enhancing detection of objects of interest or potential threats. 


2.1.2. Multisensory Integration in temporal and parietal cortical areas. 


Moving from a low-level structure of the midbrain, such as the superior collicu- 
lus, to the cerebral cortex, the properties of multisensory responses show different 
behaviours. For example, contrary to the SC, superadditivity seems to represent an 
exception rather than the rule for multisensory responses of cortical neurons. Super- 
additivity is an ideal characteristic for the guidance of orienting behaviours (that are 
the main function of the SC), for it enhances responses to regions of the external 
space showing spatio-temporal coincidence. Most cortical multisensory areas, in- 
stead have more complex functions, related to action, language, learning or even so- 
cial perception (Campanella & Belin, 2007) and are therefore likely to encode coher- 
ent objects and semantic information. Congruency of multisensory input is there- 
fore the most important feature for cortical neuronal population representing multi- 
sensory objects, more than needing a response enhancement to improve object de- 
tection and response speed to weak stimuli. 

Neurons having overlapping receptive fields from different sensory modalities 
have been found in the anterior ectosylvian sulcus of the cat (Wallace, Meredith, & 
Stein, 1992), these multisensory neurons have larger receptive fields responding to 
congruent stimuli and only a few of them shows non-linear response patterns com- 
pared to the SC (Stein & Wallace, 1996). In the primate brain, research has focused 
mainly on posterior parietal cortex (PPC) and on superior temporal areas. PPC is a 
structure involved in attention and goal directed behaviours, such as reaching and 
gaze direction. Multisensory spatial maps found in PPC are encoded in coordinate 
frames that are common for the different sensory modalities (Avillac, Deneve, Olivi- 
er, Pouget, & Duhamel, 2005) and that dynamically realign with changes in gaze di- 
rection (Y. E. Cohen, Batista, & Andersen, 2002). Only one study found non-linear 
responses of neurons of a particular region of primate PPC: neurons in the ventral 
part of the intra-parietal sulcus (VIP) respond to spatio-temporally coincident visual 
and tactile stimuli and show both super- and sub-additive behaviours (Avillac, Ben 
Hamed, & Duhamel, 2007). 

Superior temporal areas processing multisensory stimuli have been found in 
humans using neuroimaging techniques, such as fMRI. Most of those studies have 
failed in finding activations strong enough to be considered super-addictive, but 
have nevertheless reported reliable modulations of BOLD responses to multisensory 
stimuli (Beauchamp, Lee, Argall, & Martin, 2004; Beauchamp, Yasar, Frye, & Ro, 
2008; Hein et al., 2007; van Atteveldt, Formisano, Blomert, & Goebel, 2007). Only a 
couple of studies have shown super-addictive BOLD responses, for example, Foxe et 
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al (2002) have demonstrated superadditivity in the superior temporal gyrus of hu- 
mans in response to audio-tactile stimuli (Foxe et al., 2002) and Calvert, Campbell 
and Brammer (2000) have found super-addictive responses to synchronized audio- 
visual speech in the left superior temporal sulcus (Calvert, Campbell, & Brammer, 
2000). The only fMRI study that varied the intensity of auditory and visual stimuli in 
order to investigate the principle of “inverse effectiveness” in human superior tem- 
poral sulcus, found super-addictive modulation of the BOLD signal for weak stimuli 
representing simultaneous speech (Stevenson & James, 2009). 


2.1.3. Cross-modal interactions in (previously thought) unisensory areas. 


Experimental evidence listed in the previous paragraph, confirms the traditional 
view that considered multisensory integration as a process occurring at high levels of 
analysis (temporal and parietal areas) only after the accomplishment of unisensory 
processing. During the last decade this view has been challenged, with growing evi- 
dence showing a role of those that were believed to be unisensory areas and even 
primary sensory areas in multisensory processing. The growing number of cerebral 
areas showing multisensory properties has brought some authors to the provocative 
proposal of considering the whole cortex as multisensory and not as a set of separate 
areas specialized for a sensory modality (Ghazanfar & Schroeder, 2006). 

One early study by Calvert et al (1997) showed the activation (BOLD) of audito- 
ry areas in response to lipreading in absence of sound, activation that was demon- 
strated to be specific for visual lipreading of speech and not for generic facial expres- 
sions or lip movements (Calvert et al., 1997). In the same year, Sathian et al (1997), 
using PET, showed an activation of visual extrastriate areas during a tactile task in- 
volving spatial frequency discrimination of grooved gratings (Sathian, Zangaladze, 
Hoffman, & Grafton, 1997). Two years later, the same group (Zangaladze, Epstein, 
Grafton, & Sathian, 1999) also demonstrated that the visual activity elicited by tactile 
stimulation was not an epiphenomenon, but was actually useful for the performance 
of the tactile task. Inhibiting the PET-defined visual area that was activated during 
the tactile task through transcranial magnetic stimulation (TMS) impaired observ- 
er’s performance in the tactile spatial frequency discrimination task (Zangaladze, 
Epstein, Grafton, & Sathian, 1999). More recently, the primary visual cortex has 
been shown to be recruited for tactile stimulation after prolonged practice in a tactile 
task in expert Mah-Jong players (Saito, Okada, Honda, Yonekura, & Sadato, 2006) 
and a combination of activation and deactivation in early visual areas (BOLD modu- 
lation) has been shown (Merabet et al., 2007) during tactile stimulation (exploration 
of raised-dots patterns). Single-units recordings in monkeys have also revealed the 
existence of audiovisual (Ghazanfar, Maier, Hoffman, & Logothetis, 2005) and au- 
dio-somatosensory (Fu et al., 2003; Schroeder et al., 2001) interactions in the audito- 
ry cortex. 

More insight about cross-sensory interactions in early sensory areas has been 
obtained by investigating the consequences of sensory deprivation. In fact, primary 
visual cortex of early blinds is activated during auditory (Kujala et al., 1995), tactile 
(Goyal, Hansen, & Blakemore, 2006; Sadato et al., 1996) and verbal tasks (Amedi, 
Raz, Pianka, Malach, & Zohary, 2003; Burton, Snyder, Diamond, & Raichle, 2002) 
and similarly in the deaf primary auditory cortex is activated during visual tasks 
(Finney, Fine, & Dobkins, 2001). Interestingly, the activation of V1 during Braille 
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character discrimination (Sadato et al., 1996) has been demonstrated to be also func- 
tionally relevant, for the disruption of this activity impairs recognition performance. 
A very unfortunate case reported by Hamilton et al (2000) revealed the importance 
of V1 activity in tactile perception of early blinds: following an ischemic stroke that 
provoked bilateral occipital lesions, a woman blind from birth that was a proficient 
Braille reader lost her capacity of recognizing Braille characters despite preserved 
somatosensory perception (Hamilton, Keenan, Catala, & Pascual-Leone, 2000). Sim- 
ilarly, Cohen et al (1997) demonstrated that transiently disrupting occipital activity 
in early blinds with TMS impaired their ability of discriminating Braille characters 
and embossed letters, as well as their general somatosensory perception (Cohen et 
al., 1997). 

Occipital cross-modal plasticity following visual loss has been demonstrated to 
be functionally relevant also for higher order tasks, especially for linguistic pro- 
cessing. Amedi et al (2004) showed that transcranic magnetic stimulation of occipi- 
tal cortex in blinds impaired their verbal abilities (Amedi, Floel, Knecht, Zohary, & 
Cohen, 2004), a result that corroborated the hypothesis of the authors that cross- 
modal plasticity following sensory loss reflected an expansion of ventral cortex acti- 
vation to primary visual areas, with a particular region in the occipito-temporal cor- 
tex dedicated to object integration and semantic processing sending feedback to V1 
(Amedi, Jacobson, Hendler, Malach, & Zohary, 2002; Amedi, Malach, Hendler, 
Peled, & Zohary, 2001). Coherently with this view, in another study, Amedi et al 
(2003) found a left lateralized activation of the occipital cortex of blind patients dur- 
ing verbal tasks, independently of the sensory modality, and even for retrieval from 
memory (Amedi, Raz, Pianka, Malach, & Zohary, 2003). The authors also demon- 
strated that the amplitude of the occipital modulation (BOLD) during the verbal task 
positively correlated with individual observers’ verbal memory (Amedi, Raz, Pianka, 
Malach, & Zohary, 2003). Another group expanded these findings by showing that 
V1 activation of blind patients for speech analysis correlated with difficulty of the 
verbal task (Roder, Stock, Bien, Neville, & Rosler, 2002). 

The experimental evidence listed above clearly demonstrates that plastic changes 
occur in the deafferented visual cortex of blind patients, that is recruited by other 
sensory modalities. How does this plasticity occur? Are the connections between au- 
ditory and somatosensory areas to V1 created ex novo or are they reinforced follow- 
ing sensory loss? To investigate this issue, Kauffman et al (2002) measured tactile 
performance in a Braille characters discrimination task in normally-sighted partici- 
pants, before, during and after five days of blindfolding; the idea being that transient 
visual deprivation could unmask the somatosensory connections to the visual cortex. 
During the five days of blindfolding, a group of participants performed 2 hours of 
tactile training per day and two control group of non-blinfolded participants were 
also tested. After blindfolding, Kauffman et al (2002) found that tactile performance 
was significantly better for the blindfolded group independently of the tactile train- 
ing (Kauffman, Theoret, & Pascual-Leone, 2002). The authors interpreted these re- 
sults as evidence of existing somatosensory connections to visual cortex that are 
masked by the strong and reliable visual input in normally-sighted individuals. 

The same group confirmed this result in a following experiment in which they 
repeated the blindfolding procedure (Figure 2.1.2A) and the tactile discrimination 
task, but they also performed fMRI scans during and after blindfolding (Merabet et 
al., 2008). Figure 2.1.2. reports the results of the study. As in the first study, blind- 
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folded participants showed a better performance in the tactile discrimination task 
compared to non-blindfolded individuals after five days of visual deprivation. Inter- 
estingly, this improvement disappeared one day following re-exposure to normal vi- 
sion, showing a reversible recruitment of visual cortex by touch once vision is re- 
stored. Merabet et al (2008) also found that the advance in Braille character discrim- 
ination gained during blindfolding was accompanied with an increase activation 
(BOLD) in primary visual cortex during tactile stimulation compared with the con- 
trol group of non-blinfolded that underwent the same tactile training (Figure 
2.1.2C). One day following blindfolding removal, the increase in V1 activity disap- 
peared with the perceptual advantage (Figure 2.1.2D). As for blind patients, Merabet 
et al (2008) demonstrated that the recruitment of V1 by touch following blindfolding 
was functionally relevant, for inhibiting occipital cortex with repeated transcranial 
stimulation at 1Hz annulled the improvement in tactile discrimination gained by the 
blindfolded individuals at day 5 from the onset of deprivation (Figure 2.1.2B). From 
these results the authors concluded that the cross-modal plasticity observed in blind 
patients with V1 being recruited by other sensory modalities results from reinforcing 
pre-existing connections from somatosensory and auditory areas, connections that 
are present in the normally sighted, but are masked by the visual input. 
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Figure 2.1.2. Rapid and reversible recruitment of early visual cortex by touch after 5 days of 
visual deprivation. Adapted from (Merabet et al., 2008). 

Normal sighted individuals were blindfolded for five days (A) during which they underwent 
tactile training. Following blindfolding their performance to a Braille characters discrimina- 
tion task significantly improved and concomitantly V1 activation (BOLD) increased during 
tactile stimulation compared with the non-blindfolded control group (C-D). V1 recruitment 
by touch was functionally relevant: when rTMS was delivered to occipital cortex the ad- 
vantage in tactile discrimination was annulled (B). One day following re-exposure to normal 
vision their performance went back to that of non-blindfolded and the stronger activation of 
V1 disappeared (D). 


That some connections to primary visual cortex from other sensory modalities exist 
has been confirmed by anatomical studies in the monkey brain. Using a back tracer from 
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V1, Falchier et al (2002) demonstrated the existence of connections from the belt and 
parabelt auditory cortices to the region of V1 representing the peripheral visual field (10- 
20°) and connection from the polisensory area of the temporal lobe (STP) to V1 (Fal- 
chier, Clavagnier, Barone, & Kennedy, 2002). The authors also showed that auditory 
connections to V1 were weaker than somatosensory ones and in a following study con- 
firmed the existence of multisensory feedbacks to V1 (Clavagnier, Falchier, & Kennedy, 
2004). The same result was obtained by Rockland and Ojima (2003) using anterograde 
tracers injected in parietal and auditory cortices (Rockland & Ojima, 2003). 


2.1.4. Maximum Likelihood Estimation model of multisensory integration. 


As pointed out in the previous paragraphs, integrating sensory signals from dif- 
ferent modalities improves perceptual quality, for no single sensory signal can pro- 
vide reliable information about the three-dimensional structure of the world in all 
circumstances. Multisensory integration therefore produces a gain of performance 
(Newell, Bulthoff, & Ernst, 2003). One of the most common models used to describe 
multisensory integration comes from the Bayesian probability theory and proposes 
that the brain combines information from different sensory modalities in a statistical 
optimal way, so that the combine outcome is the one that is more likely to be true. 
The Maximum Likelihood Estimation (MLE) model, first proposed by Ernst and 
Banks (2002) states that the combined estimate results from a linear sum of the var- 
ious unisensory signals weighted by their reliability. That human multisensory per- 
ception matches the predictions of the MLE has been confirmed by many multisen- 
sory studies involving integration of visual, auditory and tactile cues (Alais & Burr, 
2004; Bresciani, Dammeier, & Ernst, 2006; Ernst & Banks, 2002; Ernst & Bulthoff, 
2004; Gepshtein & Banks, 2003; Gori, Del Viva, Sandini, & Burr, 2008; Gori, Mazzil- 
li, Sandini, & Burr, 2011; Helbig & Ernst, 2007; Witten & Knudsen, 2005). 

According to the MLE model if $v is the estimate of an object property provided 
by vision and $7 is that provided by touch, the combined bimodal estimate $g will re- 
sult from the following formula: $3 = wv$v + wrSr , where wy is the weight assigned to 
the visual estimate and wr that assigned to the tactile estimate. The weights represent 
the relative reliability (Ry and Rr) of the single sensory modalities that are combined in 
order to provide relevant information about the attribute of interest. So, for example 
the weight to be assigned to the visual estimate will be wy = Rv / (Rv + Rr), and vice 
versa for the tactile modality, so that the unimodal weights sum to one. The reliability 
of each sensory modality in the MLE model is defined as the inverse of the variability 
of the estimate provided by that modality, so Ry = 1 / 0°v. From this derives that the 
more reliable input is the less variable, so the input with the lowest variance will drive 
bimodal perception. The MLE model therefore allows not only the prediction of the 
mean bimodal estimate $3, but also the prediction of its variance os: 1 / o’3= 1 / 0’y + 
1 / o’y. The precision of the optimally combined estimate is always higher than the 
highest precision of the unisensory estimates. This results in multisensory perception 
being more robust, for its variance is always lower than that of the individual sensory 
modalities (Ernst & Bulthoff, 2004). If one modality is more reliable than the other, the 
integrated estimate would be much closer to the more reliable within-modality esti- 
mate; this is the case of one modality “capturing” the other (Alais & Burr, 2004). The 
highest precise (the less variable) combined estimate occurs when the within-modality 
estimates are similarly reliable (Ernst & Banks, 2002). 
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A probabilistic approach to multisensory integration in which the multimodal 
combination occurs by weighting the different sensory modalities according to their 
reliability has a potential to explain the cross-modal effects observed before the pro- 
posal of the MLE model. For example, the MLE model agrees with the “modality ap- 
propriateness hypothesis” formulated by Welch and Warren (1980), according to 
which multisensory perception is dominated by the sensory modality with the high- 
est sensitivity for the attribute of interest (Welch & Warren, 1980). This hypothesis 
was meant to explain the phenomena of “visual capture” (Bertelson & Aschersleben, 
1998; Pavani, Spence, & Driver, 2000) for the spatial domain and that of “auditory 
capture” (Alais & Burr, 2003; Fendrich & Corballis, 2001; Morein-Zamir, Soto- 
Faraco, & Kingstone, 2003; Shams, Kamitani, & Shimojo, 2000) for the temporal 
domain. The MLE model explains these results by assigning a higher weight to the 
most reliable sensory modality, but it is a more flexible model for multisensory inte- 
gration, for it predicts intermediate situations in which multimodal perception is 
dominated, but not entirely driven by the most reliable modality. 

One of the most striking examples of MLE flexibility is the “reverse ventriloquism” 
demonstrated by Alais and Burr (2004). The ventriloquist effect is a well known multi- 
sensory illusion (Howard & Templeton, 1966) in which the perceived location of an 
auditory stimulus is captured by a simultaneous visual stimulus (Slutsky & Recanzone, 
2001). Even if the auditory and visual stimuli are presented too far from each other to 
be perceived as arising from the same source, the perceived location of the auditory 
stimulus will nevertheless be attracted, even if not totally captured, towards the loca- 
tion of the visual stimulus (Bertelson & Aschersleben, 1998; Welch & Warren, 1980). 
Alais and Burr (2004) demonstrated that multisensory perception of the ventriloquist 
effect was well predicted by the MLE model (Alais & Burr, 2004) and that it could be 
reversed, with the auditory location capturing the visual one, by degrading the visual 
stimulus (blur), augmenting the variance of the visual estimate and decreasing there- 
fore the reliability of the visual signal (Alais & Burr, 2004). 

During the last decade, growing experimental evidence has shown that sensory 
modalities are often integrated in a fashion that closely approximates the predictions 
of the MLE, nevertheless, optimal multisensory integration does not occur indis- 
criminately: it is limited by spatial proximity (Gepshtein, Burge, Ernst, & Banks, 
2005), by size congruency (Gepshtein & Banks, 2003) and by temporal coincidence 
(Bresciani et al., 2005; Shams, Kamitani, & Shimojo, 2000). In a few words, combina- 
tion does not occur when signals from the different sensory modalities are not per- 
ceived as arising from the same source. Wallace et al. (2004) stressed the importance 
of perceived unity for multimodal fusion (Wallace et al., 2004). Bresciani et al (2006) 
found that subjects were able to integrate signals arising from different locations be- 
cause they could see their hand touching the same object they were seeing. In this 
case, knowing that information came from the same object favoured the combina- 
tion even though signals of the different modalities came from different spatial loca- 
tions (Bresciani, Dammeier, & Ernst, 2006). 


2.1.5. Cross-modal interactions and bistable perception 
Multisensory integration can be a powerful strategy to resolve perceptual ambi- 


guities arising from a single sensory modality. The MLE indeed predicts that com- 
bining multiple sources of sensory information is essential to achieve robust percep- 
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tion of the external world (Ernst & Bulthoff, 2004), disambiguating unreliable senso- 
ry information (e.g. noisy information). Bistable visual stimuli are intrinsically am- 
biguous and could therefore be a useful tool to reveal cross-modal interactions, for 
an unambiguous signal from a different sensory modality (audition, touch or olfac- 
tion) could help in disambiguating bistable visual perception that otherwise alter- 
nates between the two equally likely views. One early study on cross-modal disam- 
biguation reported an interaction between sound and vision during the 
stream/bounce illusion, in which two disks oscillate across the screen travelling in 
opposite directions (antiphase), when the disks converge at the centre of the screen 
they are either perceived to bounce apart and move back to the starting position or 
to continue their original trajectory after a brief overlap. Sekuler et al (1997) showed 
that adding an auditory click at the time of the convergence of the two disks strongly 
biased the interpretation towards the bouncing percept, as if the two discs were im- 
pacting with each other (Sekuler, Sekuler, & Lau, 1997). 

Several studies have reported audio-visual interactions during binocular rivalry. 
Kang and Blake (2005) demonstrated that binocular rivalry between a stationary 
grating and a grating flickering at 3Hz could be biased in favour of the flickering 
grating by introducing a sound amplitude-modulated in phase with the flickering 
grating (Kang & Blake, 2005). The sound lengthened dominance durations of the 
congruent visual stimulus, but did not affect its suppression (i.e. did not shorten 
phase durations of the incongruent visual stimulus). A similar result was recently 
obtained by Chen et al (2012) using auditory stimuli semantically congruent with 
one of the rivalring images. When the outlined drawings of a car and a bird were en- 
gaged in binocular rivalry, the concomitant presentation of a soundtrack semantical- 
ly congruent with one of the images (for example the sound of birds singing), pro- 
longed durations of that image, but did not shorten its suppression (Chen, Yeh, & 
Spence, 2012). In line with these results, audio-visual interactions between speech 
and Rubin’s face-vase illusion has been shown to require conscious perception of the 
face producing the sound (Carter, Konkle, Wang, Hayward, & Moore, 2008). 

One study showed that also suppression durations of the congruent rivalring 
stimulus can be influenced by auditory signals: by using random-dot kinetograms 
(RDKs) at different levels of motion coherence to create binocular rivalry between 
optic flows moving in opposite directions, Conrad et al (2010) showed that present- 
ing a directional motion sound congruent with one or the other rivalring motion di- 
rections promoted dominance of the congruent visual stimulus both by prolonging 
its dominance durations and by curtailing its suppression (Conrad, Bartels, Kleiner, 
& Noppeney, 2010). That suppression of motion stimuli can be influenced by a con- 
gruent sound is consistent with the evidence that, as detailed in paragraph 1.2, mo- 
tion signals can escape binocular rivalry suppression. 

Somatosensory signals have also be shown to interact with bistable visual stimu- 
li. Blake et al. (2004) used as visual stimulus the “kinetic depth effect”, which occurs 
when viewing 2D parallel projections of a rotating 3D sphere, under these conditions 
perceptual reversals occur for opposite directions of rotation. Blake et al. (2004) 
demonstrated that touching an unambiguously rotating globe biased observers’ per- 
ception in favour of the tactile rotation direction, by prolonging dominance of the 
congruent direction (Blake, Sobel, & James, 2004). Interestingly, they also demon- 
strated that pre-adaptation to a tactile direction did not bias perceived visual direc- 
tion, indicating that the interaction between vision and touch only occurs for simul- 
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taneous bimodal stimulation (Blake, Sobel, & James, 2004). A similar result has been 
obtained using a visuo-haptic version of the Necker cube in which the visual inter- 
pretation congruent with the haptic configuration was favoured (Bruno, Jacomuzzi, 
Bertamini, & Meyer, 2007). Finally, vision and touch have been recently shown to 
reciprocal interact also during ambiguous motion perception in the motion quartet 
(Conrad, Vitello, & Noppeney, 2012), a form of perceptual bistability that engages 
visual as well as tactile rivalry (Carter, Konkle, Wang, Hayward, & Moore, 2008). 

In addition to somatosensory information, active movement has been found to 
interfere with the dynamics of binocular rivalry Maruya, Yang and Blake showed 
that controlling the motion direction of one of the rivalring images with a voluntary 
action promoted the perception of the controlled visual stimulus both by prolonging 
its dominance and by curtailing its suppression (Maruya, Yang, & Blake, 2007). A 
result that is consistent with the survival of the suppressed signal along the dorsal 
pathway (see paragraph 1.2). Similarly, Hu and Knill (2010) recently reported that 
kinesthetic information provided by the arm movement can help in resolving visual 
ambiguity caused by the aperture problem (Hu & Knill, 2010). 

Most cross-modal studies that have used bistable visual perception could be in- 
terpreted in principle as a shift of attention provoked by an unambiguous signal in 
another sensory modality. Most studies, in fact, have shown that cross-sensory stim- 
ulation can extend the dominance duration of the congruent rivalring stimulus, an 
effect that is similar of that of voluntary attention on binocular rivalry (Chong, Ta- 
din, & Blake, 2005; Paffen & Alais, 2012). In line with this hypothesis, van Ee et al 
(van Ee, van Boxtel, Parker, & Alais, 2009) have demonstrated that cross-modal 
stimulation (either auditory or tactile) can enhance attentional control over binocu- 
lar rivalry even though, in their paradigm, it was ineffective automatically and re- 
quired a conscious attentional act to be effective (see paragraph 1.5.1. for details). 


2.2. Touch interacts with binocular rivalry at early stages of visual processing. 


As detailed in the previous paragraphs, growing evidence is challenging the clas- 
sical view that considered multisensory convergence as occurring at high levels of 
processing in associative temporal and parietal areas only after that information 
from the different sensory modalities had been processed by the dedicated unisenso- 
ry cortices. During the last decade, in fact, cross-modal interactions have been found 
at early stages of sensory processing, and even in primary sensory cortices that had 
always been thought to be purely unisensory. 

We were interested in investigating whether touch could interact with vision at an 
early level of visual processing, for recruitment of primary visual cortex for tactile pro- 
cessing has been demonstrated in blind patients (Merabet et al., 2008; Sadato et al., 
1996) and in normally sighted individuals after few days of visual deprivation (Me- 
rabet et al., 2008). Binocular rivalry is a powerful tool to investigate this issue. The in- 
trinsic ambiguity of rivalrous visual perception creates ideal conditions to favour 
cross-modal interactions because disambiguation is thought to be one of the primary 
roles of cross-modal perception (Ernst & Bulthoff, 2004). Moreover, the temporary 
blindness to one image provoked by binocular rivalry suppression could unmask the 
somatosensory connections with early visual areas by weakening the visual signal 
through adaptation, added noise and mutual inhibition from the dominant stimulus 
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(Alais, Cass, O'Shea, & Blake, 2010). Finally, binocular rivalry suppression is thought 
to take place early in the visual system, attenuating neural activity related to the sup- 
pressed image from the earliest stages of visual analysis (LGN, Haynes, Deichmann, & 
Rees, 2005, monocular columns in V1, Tong & Engel, 2001) and completely destroy- 
ing it at higher levels of visual processing (Tong, Nakayama, Vaughan, & Kanwisher, 
1998), so that, if appropriate visual stimuli are used (for example oriented gratings), 
neural activity associated with the suppressed image is not traceable outside of V1 or 
V2. Observing an interference of the tactile signal with the suppressed visual stimulus 
will therefore point o an early locus of the cross-modal interaction. 

In the next paragraphs we will present three studies in which we investigated the 
interaction between vision and touch during binocular rivalry. In the first study we 
showed that haptic exploration of an oriented grating promoted dominance of the 
parallel visual stimulus both by prolonging its dominance and by curtailing its sup- 
pression and that the interaction was strictly tuned for matched visuo-haptic spatial 
frequencies. In the second study, we confirmed the main effect of touch and we ex- 
panded the selectivity of the interaction for orientation. By performing a cross- 
sensory orientation discrimination experiment we also demonstrated that the sharp 
orientation tuning of the visuo-haptic interaction did not result from a conscious 
categorization, ruling also out the possibility of attention and response bias playing a 
leading role in mediating the interaction. In the last experiment, we showed that spa- 
tial proximity between the visual and the haptic stimulus is necessary to observe the 
interaction and that passive tactile stimulation is as effective as active haptic explora- 
tion in influencing the dynamics of binocular rivalry, showing as well the same selec- 
tivity for spatial frequency. Finally, we also found that simultaneous presentation of 
visual and haptic stimuli is crucial for cross-sensory interaction by showing that 
haptic stimulation does not prime the onset of rivalry. 


2.2.1. Touch specifically interferes with the dynamics of binocular rivalry 


When incompatible images are presented to corresponding retinal regions, in- 
terocular differences prevent the brain from achieving binocular fusion and normal 
binocular vision is not possible. Instead, the conflicting images trigger a continual 
struggle for visual awareness in which only one image is perceived at a time and the 
other is suppressed from awareness. These perceptual alternations, a form of bistable 
perception known as binocular rivalry (Alais & Blake, 2005; Levelt, 1965), will con- 
tinue irregularly each second or two for as long as the conflicting stimuli are present. 
Because of the peculiar dissociation between a continuous physical stimulation but 
an alternating visual perception, binocular rivalry has been used as a tool to investi- 
gate the neural correlates of visual awareness and the resolution of perceptual ambi- 
guity (Klink, van Wezel, & van Ee, 2012; Koch, 2007; Logothetis, 1998). 

Binocular rivalry suppression takes place early, probably in V1 (Tong, Meng, & 
Blake, 2006). However, a trace of the suppressed signal has been detected along the dor- 
sal visual pathway (BOLD responses, Fang & He, 2005) and demonstrated with psycho- 
physical experiments. In line with the evidence of neural signals related to the suppressed 
visual stimulus surviving along the dorsal pathway, the suppressed image of a rotating 
sphere during rivalry is restored to consciousness when the observer actively controls the 
rotation (Maruya, Yang, & Blake, 2007) and a similar effect on the suppressed signal has 
been shown for motion perception (Andrews & Blakemore, 1999). 
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The resolution of perceptual ambiguity is in thought to be one of the main func- 
tions of cross-modal interactions (Ernst & Bulthoff, 2004; Klink, van Wezel, & van 
Ee, 2012), bistable perception is therefore a powerful tool to reveal an interference of 
another modality on vision. In this first experiment, that was published in 2010, we 
asked whether cross-modal sensory signals could selectively interact with rivalrous 
visual signals (oriented gratings) that are analyzed at a very early stage, probably V1. 


2.2.1.1. Materials and Methods 


Subjects 

Seven subjects (all females) participated in the first experiment and six subjects 
(one male) participated in the second experiment in which we measured spatial fre- 
quency selectivity; all had normal or corrected to normal vision, normal stereo acui- 
ty (Frisby stereotest, Sasieni, 1978), and no strong eye dominance. 


Apparatus and stimuli 

The experiment took place in a dark and quiet room. Visual stimuli were gener- 
ated by the ViSaGe (CRS, Cambridge Research Systems) housed in a PC (Dell) and 
controlled by Matlab programs. Stimuli were displayed on a 21” Sony GDM-F520 
Trinitron colour monitor driven at a resolution of 1280x800 pixels, with a refresh 
rate of 120 Hz. Subjects’ head position was stabilized by a neck-rest. They viewed the 
display at a distance of 40 cm through CRS Ferro-Magnetic shutter goggles that oc- 
cluded alternately the two eyes at the same frequency as the refresh rate of the moni- 
tor. Responses were recorded through the computer keyboard. Visual stimuli were 
two Gabor Patches, oriented either vertically or horizontally (size: 1.5°, spatial fre- 
quency: 5 c/cm, corresponding to 3.5 cpd, contrast: 45%), presented on a grey back- 
ground (luminance: 7.8 cd/m’, C.I.E: 0.299 0.327) in central vision. Presentations 
were alternated at the same frequency as the shuttering goggles, so each eye was pre- 
sented with only one of the two stimuli. The haptic stimulus was a 12.8 x 15 cm si- 
nusoidally grooved Plexiglas, of the same spatial frequency as the visual stimuli (5 
c/cm), positioned on a prop fixed on the table under the monitor. It could be rotated 
by the experimenter so that its orientation could be either vertical or horizontal, ie. 
either parallel or orthogonal to the orientation of the visual percept. Subjects could 
see neither the Plexiglas nor their hand touching it; they had no visual feedback 
while reaching the stimuli. The main task of the subjects was to report by key-press 
their visual perception, horizontal or vertical. They rubbed their right thumb over 
the grooved Plexiglas; they were not asked to pay particular attention to it, but to 
consider that it was generated by the same object that was seen through the goggles 
(even though the two stimuli were presented at different spatial locations or provid- 
ed conflicting information, a common practice in multisensory integration studies, 
e.g. Alais & Burr, 2004; Ernst & Banks, 2002; Gori, Del Viva, Sandini, & Burr, 2008). 

For the second experiment that measured the spatial frequency selectivity of the 
effect, the visual stimuli were displayed on a 20-inch Clinton Monoray (Richardson 
Electronics Ltd., LaFox, IL) monochrome monitor (which has a fast phosphor and 
therefore less leakage between the eyes), driven at a resolution of 1024x600 pixels, 
with a refresh rate of 120 Hz. Visual stimuli were of 3 c/cm (corresponding to 1.6 
cpd at a distance of 30 cm) or 1.3 c/cm (corresponding to 0.7 cpd) spatial frequency 
(background luminance: 37.4 cd/m’, C.I.E: 0.442 0.537). Haptic stimuli were 4 si- 
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nusoidal gratings of different spatial frequencies (size: 3 cm, spatial frequencies: 1.3 
c/cm, 2 c/cm, 3 c/cm, 4 c/cm) created with a 3D printer (Dimension Elite SST, Stra- 
tasys, Inc., Eden Prairie, MN), positioned on a touchscreen (Magic Touch, Keytec 
Inc., Richardson, TX), in order to record the timing of touch periods. Different spa- 
tial frequencies were presented in separate blocks in randomized order, following a 
preset table of random number generator. 


Task and procedure 

After a countdown, the visual stimuli appeared generating binocular rivalry 
(Levelt, 1965). Subjects reported their visual perception (vertical or horizontal) by 
continuously pressing with the left hand one of two keys (left or right arrows) of the 
computer keyboard. As assessed in pilot studies and in debriefing sessions, given the 
small size of the stimuli, mixed percepts occurred for very brief periods, only during 
perceptual transitions, and their frequency remained constant across conditions. 
During the observing period, at approximately regular intervals, subjects were asked 
to explore the haptic stimulus with the right thumb performing circular movements 
(in order to avoid a possible influence of the motor direction), until the experiment- 
er gave a stop signal (each touch period had a mean duration of 3 s and a standard 
deviation of 0.7); touch periods were interleaved with no-touch (vision only) periods 
(mean duration 7.1 s, standard deviation 1.9 s). During no-touch periods, the exper- 
imenter changed the orientation of the Plexiglas, alternating the vertical and hori- 
zontal orientations following preset random generated sequences (the manoeuvre 
was invisible to the subjects and the orientation of the Plexiglas unpredictable at 
each touch period). Touch periods and Plexiglas orientation were recorded by the 
experimenter by continuously pressing the appropriate mouse button. In both ex- 
periments, sessions lasted 180 s and were divided in two parts; in the second part the 
orientation of the visual stimuli was swapped between the two eyes. 

We also recorded pilot data where the duration of haptic stimulation was ex- 
tended up to 3 minutes. The results indicated that the subjects had difficulty in keep- 
ing attention to the haptic stimulus, while attention is important in modulating bin- 
ocular rivalry (Chong & Blake, 2006) and cross-modal interactions during it. Given 
the dynamics of the effect we chose the 3s interval that optimized the number of 
sampled touch periods for each recording session. 


2.2.1.2. Results 


We tested whether touch could interfere with the dynamics of binocular rivalry 
by having subjects explore haptically a grooved stimulus of the same orientation as 
one of the two rival visual stimuli. Visual stimuli were orthogonal Gabor Patches 
(oriented vertically or horizontally), displayed alternatively to the two eyes; the hap- 
tic stimulus was a sinusoidal grooved Plexiglas, matched in spatial frequency with 
the visual gratings; subjects could see neither their hand nor the stimulus they 
touched. While reporting the perceived orientation of the visual stimulus, partici- 
pants were required at random intervals to touch (with circular movements of the 
right thumb, to avoid interference of movement direction) the haptic stimulus and 
to release it after a brief period of ~3 sec (short enough to avoid deployment of at- 
tention (van Ee, van Boxtel, Parker, & Alais, 2009) and adaptation to the haptic 
stimulus, but long enough to allow tracing its effect on the slow dynamic of binocu- 
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lar rivalry). For each touch period, visual and haptic stimuli were unpredictably 
parallel to each other or orthogonal (figure 2.2.1.1.A). 
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Figure 2.2.1.1. Experimental paradigm and main results. 

The black line in Panel A represents the visual perception switching between dominance of the 
vertically and horizontally oriented visual stimulus. Two example touch periods are shown, the 
first beginning with parallel visuo-haptic stimulation and maintaining it for the whole touch 
period, the second beginning with orthogonal visuo-haptic stimulation and switching to par- 
allel. The probability of maintaining dominance of a stimulus during the whole period (Panel 
B, ** refers to p<0.01, *** to p<0.001) was significantly greater for periods of congruent visuo- 
haptic stimulation (light grey bar), both compared to periods of incongruent stimulation (dark 
grey bar) and to no touch periods (white bar). Panel D shows individual subjects data: all circles 
lie beneath the graph bisector (y < x). On the other hand, the probability of switching percep- 
tion (Panel C) was significantly greater during periods of orthogonal visuo-haptic stimulation 
both compared to periods of parallel stimulation and to no touch periods. Panel E shows indi- 
vidual subjects’ data: the effect is consistent for all subject, all circles lie over the graph bisector 
(y > x). Panel F and G show the results of a spatial frequency selectivity experiment: the prob- 
ability of switching perception when the touch period began with an orthogonal visual percept, 
is plotted as a function of the spatial frequency of the haptic stimuli (ranging from 1.3 c/cm to 
4 c/cm). The arrow represents the visual spatial frequency tested. The effect peaks when visual 
and haptic spatial frequencies are matched. (* refers to p<0.05, ** to p<0.01). 


During a touch period observers could either maintain in dominance the same 
visual stimulus for the whole touch period duration, or they could switch in favour 
of the other visual stimulus. We computed the probability of these two alternatives 
conditioned to the type of visuo-haptic stimulation (parallel, orthogonal or for peri- 
ods of visual-only stimulation of comparable duration) and we found that the prob- 
ability of maintaining or switching perception was importantly influenced by touch. 
Haptic stimulation disambiguated visual perception both by promoting dominance 
of the visual stimulus with the same orientation and by reducing suppression. 

When the orientations of the haptic and visual stimuli were orthogonal, the 
probability of switching perception, re-establishing congruency of visual and haptic 
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stimuli orientation, increased by 35% relative to periods of congruent visuo-haptic 
stimulation and by 20% relative to no touch periods (figure 2.2.1.1.C). This suggests 
that the haptic signal boosted the suppressed stimulus signal provoking a perceptual 
switch, leading the parallel stimulus to revert to perceptual dominance. Conversely, 
when haptic and visual stimuli were parallel, the probability of maintaining domi- 
nance of the visual stimulus during the whole period of haptic stimulation increased 
by 34% relative to periods of orthogonal visuo-haptic stimulation and by 20% rela- 
tive to no touch periods (figure 2.2.1.1.B). Hence the duration of dominance as well 
as of the suppression of a rival visual stimulus can be substantially affected by the 
presentation of a congruent haptic stimulus. The individual data reported in Figures 
figure 2.2.1.1.D and figure 2.2.1.1.E show that the effect is consistently observed in 
all tested subjects. 


Time of first switch (sec) 


Figure 2.2.1.2. Effect of haptic stimulation on dominance durations. 

The three bars show the time, averaged across subjects, of the first switch relative to the onset 
of a touch period for the three tested conditions: parallel (light grey), orthogonal (dark grey) 
visuo-haptic stimulation, or visual-only stimulation (white bar). The horizontal dashed line 
represents the average duration of touch periods (3.01 sec). The first switch occurred later for 
parallel visuo-haptic stimulation than for orthogonal visuo-haptic stimulation. The difference 
between these two values was statistically significant (p<0.001, three stars), as well as the differ- 
ence between each one and the no-touch value (p<0.05, one star). 


Since touch periods were delivered at random intervals during binocular rivalry 
viewing the onset of touch never coincided with the onset of a dominance phase, we 
could not compute mean phase durations of one or the other visual stimulus during 
haptic stimulation. In order to quantify the effect on dominance durations we there- 
fore computed the time of the first switch following the onset of touch separately for 
touch periods in which haptic stimulation was parallel to the dominant visual stimu- 
lus, for periods in which the haptic orientation was orthogonal to the visual orienta- 
tion and for no-touch periods. In agreement with the probability pattern, we found 
that haptic stimulation prolonged dominance duration of the parallel visual stimu- 
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lus and curtailed dominance duration of the orthogonal visual stimulus compared to 
no-touch periods (figure 2.2.1.2.). For parallel visuo-haptic stimulation the first 
switch exceeded the offset of touch and occurred on average at 3.94+0.48 sec from 
touch onset. Vice versa, for orthogonal visuo-haptic stimulation the first switch oc- 
curred well before the offset of touch (mean time: 2.6+0.3 sec). 

By repeating the experiment using visual and haptic stimuli of different spatial 
frequencies we found that the effect of touch on the suppressed visual stimulus was 
selective for matched visuo-haptic spatial frequencies. Figures 1 figure 2.2.1.1.F and 
figure 2.2.1.1.G show the modulation of the probability of switching perception from 
the orthogonal visual stimulus towards the haptic orientation as a function of the 
haptic spatial frequency for two different spatial frequencies of the visual stimuli. 
The probability peaks when the visual and the haptic stimuli have the same spatial 
frequency. 


2.2.1.3. Discussion 


Our results demonstrate that a visual stimulus, rendered invisible by binocular 
rivalry suppression, can nonetheless revert to consciousness when boosted by a con- 
comitant haptic signal of congruent orientation and spatial frequency. When there is 
a conflict between the orientation of the visual and the haptic stimuli, vision switches 
to the congruent orientation, indicating that the system uses the haptic signal to re- 
solve binocular rivalry. Touch and visual signals are optimally fused in many tasks 
(Hillis, Ernst, Banks, & Landy, 2002), but the fusion is not mandatory, consistent 
with the prevailing view that it is mediated by high-level associative multisensory 
cortex. In previous studies, for bistable visual signals, the haptic stimulus cannot 
modulate the suppressed phase (Blake, Sobel, & James, 2004) and active listening 
was required to increase the phase duration of the congruent visual stimulus (van Ee, 
van Boxtel, Parker, & Alais, 2009). However, the present results show that touch can 
make visible an otherwise unseen stimulus, suggesting mandatory fusion. 

That touch influenced binocular rivalry both by prolonging dominance of the 
parallel visual stimulus and by shortening its suppression (i.e. shortening dominance 
of the orthogonal visual stimulus) was confirmed when we computed the time of the 
first switch from the onset of touch. This indicates that touch interferes with the 
suppressed oriented grating, shortening the suppressed phase and reverting the par- 
allel orientation stimulus to dominance. Interestingly, several psychophysical (Free- 
man, Nguyen, & Alais, 2005) and neurophysiological (Tong, 2005) studies have 
shown that neural activity related to the suppressed visual stimulus is disrupted be- 
yond V1. Suppression of oriented gratings during binocular rivalry takes places after 
orientation adaptation (Blake & Fox, 1974a) and orientation fusion (Pearson & 
Clifford, 2005a), two phenomena that are typical of V1 circuitry, while adaptation to 
more complex stimuli, like faces, is abolished during suppression (Moradi, Koch, & 
Shimojo, 2005). Nevertheless, a residual trace of the suppressed stimulus can be de- 
tected beyond V1. 

With one exception (Sterzer, Jalkanen, & Rees, 2009), evidence suggests that 
such residual activities occur along the dorsal pathway (Fang & He, 2005) which is 
known to mediate action rather than conscious perception (Goodale & Milner, 
1992). This residual trace may interact with visible signals. Using plaid motion stim- 
uli, it has been shown that the suppressed signal changes the perceived direction of 


56 


Claudia Lunghi 


motion (Alais & Parker, 2006; Andrews & Blakemore, 1999) and the reflexive eye 
movement associated with it (Spering, Pomplun, & Carrasco, 2009). Interestingly, in 
a different context where binocular rivalry was not involved, a non informative tac- 
tile stimulus was shown to make detectable an otherwise under-threshold visual 
stimulus (Arabzadeh, Clifford, & Harris, 2008). These evidences, together with the 
spatial frequency selectivity of the effect on the suppressed signal reported here and 
in the main text, strongly suggest that touch of simple oriented gratings interact with 
vision at early level, probably at V1, in agreement with recent neurophysiological 
data (Ghazanfar & Schroeder, 2006). 

In principle, the effect could be mediated by attention or imagination. However, 
the spatial frequency selectivity of the observed effect (with the peak effect changing 
with the spatial frequency of the visual stimulus) is inconsistent with attentive or 
other non-specific effects, and points strongly to early interactions between haptic 
and visual signals. V1 neurons have the narrowest spatial frequency tuning of visual 
neurons and could mediate the effect reported here, which requires a bandwidth of 
less than 1 octave. 

Few factors decrease suppression of a rival stimulus: only contrast or, to a much 
lesser degree, attention (Paffen, Alais, & Verstraten, 2006). Interestingly, both the 
contrast of the stimulus and attention are known to modulate primary visual cortex 
activity. In order to boost a suppressed visual signal, therefore, it is very likely that 
the haptic signal must reach early visual cortex, in line with previous evidence from 
blind patients, in which VI is recruited for tactile processing (Pascual-Leone, Amedi, 
Fregni, & Merabet, 2005). It has recently been proposed that reorganization of the 
visual cortex in the blind reinforces pre-existing connections between somatosenso- 
ry and visual cortex, the tactile input to visual cortex being normally masked by the 
robust visual input. In the absence of visual inputs (for example during prolonged 
blindfolding (Pascual-Leone, Amedi, Fregni, & Merabet, 2005)), these connections 
become more effective. We propose that the same happens during binocular rivalry 
suppression, revealing the non-visual roles of the occipital cortex by unmasking tac- 
tile inputs. Our findings strongly suggest that not only can haptic signals modulate 
activity at the earliest stages of visual processing (consistent with recent neurophysi- 
ological evidence, see Supplemental References), but that these signals influence low 
level visual perception opening the way to consciousness for an otherwise invisible 
stimulus. 


2.2.2. The interaction between vision and touch during binocular rivalry is strict- 
ly tuned for orientation 


In our previous experiment (Lunghi, Binda, & Morrone, 2010) we have shown 
that touching an oriented haptic grating congruent with one or the other rivaling 
images extended the dominance duration of the parallel visual stimulus. In addition, 
and unlike the effects of attention on rivalry, we have demonstrated that the tactile 
signal shortened the period of suppression of the congruent visual signal and re- 
stored it more quickly to consciousness than in a visual-only condition. This implies 
that the tactile signal interacts with the congruent visual signal outside of awareness 
(i.e., while it is suppressed). We have also shown that the influence of haptic stimula- 
tion on the dynamics of binocular rivalry was tuned for spatial frequency in that vis- 
ual and tactile gratings with the same spatial frequency produced an effect whereas a 
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difference of one octave eliminated it. From these observations (haptic interaction 
during suppression and spatial frequency tuning), and given that neural signals as- 
sociated with the suppressed image are not traceable outside of V1-V2 (Blake & 
Logothetis, 2002; Lin & He, 2009) and that neurons with narrow spatial tunings are 
only found in primary visual cortex (Maffei & Fiorentini, 1973), we interpreted our 
data as evidence of an early, compulsory interaction between vision and touch. 

The conclusion that vision and touch interact early in visual processing squares with 
other recent evidence. Multisensory convergence in primary visual cortex has been 
found in the macaque brain (Falchier, Clavagnier, Barone, & Kennedy, 2002), and re- 
cently the primary visual cortex of rodents has been shown to respond to tactile explora- 
tion of novel objects (Vasconcelos et al., 2011), its response correlating with the animal’s 
performance in a tactile aperture discrimination task. In humans, V1 activity (BOLD) 
has been found in response to tactile stimulation (Merabet et al., 2007), moreover, the 
primary visual cortex is recruited for tactile processing both in blind patients (Sadato et 
al., 1996) and in blindfolded normal-sighted adults (Merabet et al., 2008). The early visu- 
al-touch interaction fits with recent evidence of multisensory convergence and with a 
proposal that the whole brain is fundamentally multisensory, including areas traditional- 
ly thought of as primary sensory cortices (Ghazanfar & Schroeder, 2006). 

During our visiting period at the School of Psychology of the University of Syd- 
ney, we have further investigated the hypothesized early interaction between vision 
and touch during binocular rivalry by testing whether it is specific for another basic 
property of the primary visual cortex, orientation selectivity (Hubel, Wiesel, & 
Stryker, 1977). Using the same paradigm as the first experiment, we confirmed that 
the effect of congruent touch in binocular rivalry acts in two ways, both extending 
the dominance duration of the visible stimulus and reducing the time that the invisi- 
ble stimulus is suppressed. Importantly, by varying the relative orientation of the 
visual and tactile gratings, we show that this effect of touch on vision is strictly ori- 
entation tuned, reinforcing the hypothesis that the interaction occurs at the earliest 
stages of visual analysis, probably V1. 


2.2.2.1. Materials and Methods 


Subjects 

Eight subjects (four males, average age 28.3+7.3 years), including the authors, par- 
ticipated in the main experiment (subject JT took part in only five conditions) and five 
subjects from this group also participated in the orientation discrimination experi- 
ment; all had normal or corrected-to-normal vision and no strong eye preference. 

Ethics Statement 

Participants gave written informed consent. The experimental procedure con- 
formed to the declaration of Helsinki and was approved by the local ethics commit- 
tee (Human Research Ethics Committee (HREC) Low Risk Executive Committee, 
University of Sydney, Protocol No. 14893). 


Apparatus and Stimuli 

The experiment took place in a dark, quiet room. Visual stimuli were created in 
MATLAB using PsychToolbox (Brainard, 1997), and displayed on a 17-inch LCD 
monitor (Hp 1702), driven at a resolution of 1280x1024 pixels with 60 Hz refresh 
rate . Observers viewed the visual stimuli through a mirror stereoscope placed 40 cm 


58 


Claudia Lunghi 


from the monitor. In the main experiment responses were recorded through the 
computer keyboard, in the cross-modal orientation discrimination responses were 
also recorded through a pedal switch. Visual stimuli were two oblique orthogonal 
achromatic gratings (orientation: +45°, size: 2.5 cm, spatial frequency 2 cyc/cm, con- 
trast 20%, mean luminance 48 cd/m’), surrounded by a white smoothed circle in- 
cluded in a white squared frame (size 3.6 cm) to facilitate binocular fusion, presented 
on a black uniform background (luminance 0.28 cd/m?) in central vision. The tactile 
stimulus was a sinusoidal grating (size: 3 cm, spatial frequency 2 cyc/cm) created 
with a 3D printer (a diagram of the experimental setup is reported in Figure 
2.2.2.1A). Participants could not see their hand or the tactile stimulus during the ex- 
periment. The tactile grating was attached to a shaft and its orientation could be pre- 
cisely varied by the experimenter using a calibrated switch. 


Task and Procedure 
Binocular Rivalry 


The dynamics of binocular rivalry can be influenced by haptic signals, as previ- 
ously documented (Lunghi, Binda, & Morrone, 2010). Here we investigate the orien- 
tation tuning of this effect of touch on vision during binocular rivalry. Using the 
same experimental paradigm as Lunghi et al (Lunghi, Binda, & Morrone, 2010), we 
varied the orientation of the haptic stimuli in different experimental blocks, intro- 
ducing a mismatch in orientation with the visual stimuli ranging from +7.5° to +30°. 

Each observer participated in five 180-sec experimental sessions x seven experi- 
mental conditions for a total time of 105 minutes, over different days. Seven differ- 
ent haptic conditions in which a mismatch in orientation between the visual and 
haptic stimuli was introduced were tested in separate blocks. Within every block two 
orthogonal haptic orientations were tested, one clockwise, the other counterclock- 
wise relative to vertical. The mismatch in orientation was defined as the difference 
between the orientation of the visual gratings (+45°) and the haptic gratings whose 
orientation could be: -30°, -15°, -7.5°, 0°, +7.5°, +15°, or +30° relative to the orienta- 
tion of the visual. The order of the different conditions was randomized for every 
observer. The two haptic gratings in a block were always orthogonal to each other. 
Within each block, this grating pair was rotated by a fixed amount relative to the 
visual gratings, so that one haptic grating was offset by that amount relative to one 
visual grating and the other haptic grating was offset by the same amount relative to 
the other visual grating. 

Observers were given time to adjust the stereoscope in order to achieve perfect 
binocular fusion at the beginning of every experimental block when only the square 
frames were presented. When ready, observers pressed a key and after an acoustic 
signal (beep) the visual stimuli appeared. Participants were instructed to report their 
rivalry fluctuations by indicating continuously which visual orientation (clockwise 
or counterclockwise) they perceived. They did so by pressing one of two keys on the 
computer keyboard. With the small stimuli we used, the rivalry percepts in our ex- 
periment were generally coherent and unitary, with mixed perception occurring on- 
ly briefly at the time of percept transitions. At the end of each experimental session 
the orientation of the rival stimuli were swapped between the eyes to counterbalance 
any eye dominance effects. 
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During the 3-minute trials, at approximately regular intervals, observers were asked 
to explore the haptic stimulus with their right thumb by performing a constant transla- 
tional movement, until the experimenter gave a stop signal (average touch period: 
2.65+0.18 s). The experimenter manipulated the orientation of the haptic stimulus be- 
tween each touch period by alternating between orthogonal orientations (clockwise or 
counterclockwise) according to a pre-computed random sequence so that the haptic ori- 
entation was unpredictable. A cartoon of the experimental paradigm is reported in Fig- 
ure 1B. Touch periods were brief to avoid haptic adaptation and maintain the haptic 
stimulation salient. Touch periods were compared to no-touch control periods. Since the 
effect of haptic stimulation takes more than 1 second to recover after the offset of touch, 
we defined the no-touch control periods as starting 1.8 seconds after the offset of touch 
and matched their duration to that of the touch periods (i.e., 2.65 s). 


Cross-modal orientation discrimination 

In order to evaluate if observers were aware of the orientation mismatch be- 
tween the visual and the haptic stimuli we ran a cross-modal orientation discrimina- 
tion experiment. During simulated binocular rivalry, we varied the orientation dif- 
ference between the visual and the haptic stimuli and asked observers to judge 
whether the visual and the haptic stimuli had the same orientation or not. 

As in the rivalry experiment, observers in the orientation discrimination exper- 
iment were given time to adjust the stereoscope in order to achieve perfect binocular 
fusion at the beginning of every experimental block. When ready, observers initiated 
the trial sequence with a key-press and the visual stimuli appeared. Binocular rivalry 
was simulated by binocularly presenting iso-oriented gratings (either +45° or -45° 
relative to vertical) for durations randomly varying between 2 and 3.5 seconds. To 
simulate the brief patchwork of gratings often perceived during dominance transi- 
tions both orthogonally oriented gratings (+45° and -45°, Guzman-Martinez, Orte- 
ga, Grabowecky, Mossbridge, & Suzuki, 2012a) were presented for a short random 
duration of 0.5-0.7 s. All transitions from one grating to patchwork to the other grat- 
ing were temporally smoothed and observers were not told that it was a rivalry simu- 
lation and in debriefing sessions none of them reported noticing it was a rivalry 
mimic condition. Observers tracked their visual perception by continuous key-press 
as in the first experiment. At approximately regular intervals observers were asked to 
explore with their right thumb the haptic stimulus that in a given each touch period 
could have either the same orientation as the visual stimuli (+45° or -45°) or could 
be mismatched by -7.5°, -15°, +7.5° or +15° relative to the visual stimulus. All clock- 
wise and all counterclockwise haptic orientations were tested in separate blocks. 
During the touch period (mean touch duration 2.1+1.1 s), observers were required 
to indicate whether the haptic stimulus was further clockwise or further counter- 
clockwise relative to the visual stimulus, and they did so by pressing the appropriate 
pedal on a two-pedal floor switch. Only responses registered during the touch period 
were considered for analysis, meaning in effect that observers had to make their re- 
sponse within about 2 seconds of commencing their touch exploration. In order to 
validly compare discrimination of the visual-haptic orientation difference across 
conditions, we considered for analyses only haptic orientations that were tilted “fur- 
ther clockwise” than the visual orientation. This was because performance for the 0° 
visual-haptic difference showed a bias towards clockwise responses which would 
spuriously exaggerate differences with haptic gratings tilted counterclockwise. Ob- 
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servers’ performance for haptic orientations tilted “further counterclockwise” did 
not statistically differ from performances for “further clockwise orientations”. 


2.2.2.2. Results 


Observers reported binocular rivalry alternations between orthogonal gratings 
while periodically exploring an adjacent haptic grating (Figure 2.2.2.1A) which was 
randomly varied by the experimenter to be either parallel or orthogonal to the visual 
stimulus dominating the observer’s perception at that time (Figure 2.2.2.1B). During 
a touch period observers could either maintain the same visual percept or switch 
perception in favour of the previously suppressed visual stimulus (Figure 2.2.2.1B), 
or in a minority of cases more than one switch might occur. We therefore computed 
the probabilities of switching visual percept, of maintaining the same percept, or of 
switching more than once during the touch period (conditioned to the type of visuo- 
haptic stimulation: parallel, orthogonal or for a no-touch control period of visual- 
only stimulation). If haptic stimulation interacts with binocular rivalry dynamics by 
promoting dominance of the parallel visual stimulus, we would expect the probabil- 
ity of maintaining the same visual percept for the touch period to be higher for par- 
allel visuo-haptic stimulation, or the probability of switching to be higher for or- 
thogonal visuo-haptic stimulation (or both, as in our previous study). 

We found that haptic stimulation promoted dominance of the parallel visual 
percept both by increasing its dominance and by curtailing its suppression (replicat- 
ing the results obtained in the previous study). When the visual and haptic stimuli 
were parallel, the probability of maintaining the same visual percept during the 
whole touch period increased by 30% relative to orthogonal visuo-haptic stimula- 
tion, and by 21% relative to no-touch control periods. Conversely, when the visual 
and haptic stimuli were orthogonal, the probability of switching visual percept in- 
creased by 37% relative to parallel visuo-haptic stimulation and by 19% compared to 
no-touch periods (Figure 1C, statistics are reported in the figure caption). These re- 
sults demonstrate that touch specifically interacts with vision during binocular rival- 
ry both by maintaining congruency between the visual and haptic stimuli when the 
haptic stimulus is parallel to the dominant visual stimulus (retarding rivalry alterna- 
tions to the orthogonal stimulus in the other eye) and by re-establishing dominance 
of congruent visual stimulus when it is rendered invisible by binocular rivalry sup- 
pression (boosting the suppressed visual stimulus and reverting it to consciousness). 

To examine the time-course of the influence of haptic stimulation on the dynam- 
ics of binocular rivalry, we computed the instantaneous probability of perceiving the 
visual grating parallel to the haptic stimulus as a function of time from the onset of a 
touch period (collapsing parallel and orthogonal visuo-haptic stimulation). If the two 
rival stimuli are equally likely to be perceived, the probability trace oscillates around 
chance level. We found that 1 second after the onset of touch the probability trace is 
significantly biased towards the visual percept parallel to the haptic stimulus (Figure 
2.2.2.1D). The effect peaks at 2.35 seconds and slowly decays to chance level at around 
4 seconds after the onset of haptic stimulation (1.4 seconds after its offset). 

The results presented thus far were for haptic gratings that were either parallel or 
orthogonal to the dominant visual percept and show a significant effect of touch on 
vision. Is the effect of touch on vision more finely tuned for orientation? Figure 2 
plots the probabilities for maintaining visual percept (left-hand panel) and for 
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switching visual percept (right-hand panel) when the haptic stimulus was mis- 
matched in orientation relative to the visual stimuli by +7.5°, +15° or +30°. Unlike 
the results for a 0° orientation difference reported above, the data in Figure 2 show 
clearly that neither the probability for maintaining (Figure2.2.2.2A) nor the proba- 
bility for switching (Figure 2.2.2.2B) differed from the no-touch control periods 
when there was a relative orientation difference between the visual and haptic stimu- 
li (statistics reported in the figure caption). While both probabilities peak when visu- 
al and haptic stimuli are perfectly matched in orientation (0° difference), the effects 
of touch at +7.5°, +15° and +30° were not significant for either switching or main- 
taining. This demonstrates that the interaction between vision and touch during 
binocular rivalry is strictly orientation tuned. 
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Figure 2.2.2.1. Experimental design and results for matched visuo-haptic orientations. 

Panel A shows a diagram of the experimental setup: orthogonal gratings (+45°) were present- 
ed separately to the eyes through a mirror stereoscope. During binocular rivalry, at random 
intervals, observers were asked to explore the haptic stimulus with the right thumb. Panel B 
shows a diagram of the experimental paradigm that was identical to that of our first study. The 
probability of switching, maintaining or switching more than once conditioned to the type of 
visuohaptic stimulation (parallel, orthogonal or no-touch) is plotted in Panel C. When visuo- 
haptic stimulation was orthogonal, the probability of switching (dark grey bars) significantly 
increased compared both to parallel stimulation (+37%) and no touch periods (+19%). Con- 
versely, the probability of maintaining (light grey bars) significantly increased for parallel visuo- 
haptic stimulation, both compared to orthogonal stimulation (+30%) and to no-touch periods 
(paired samples t-test, two tailed, a = 0.05, N=8, **p<0.01, *p<0.05 ). The time-course of the ef- 
fect of touch on binocular rivalry is reported in Panel D, where the average instantaneous prob- 
ability of perceiving the visual stimulus parallel to the haptic orientation (black line) is plotted 
as a function of time elapsed from the onset of touch (the grey thin lines represents individual 
observers traces). The probability trace is significantly biased in favour of the haptic orientation 
1.05 seconds after the onset of touch (t-test, two tailed, a = 0.05, N=8), the probability peaks 
2.35 seconds after the onset of touch and slowly decays to chance level after the offset of touch 
(the thick red line represents the average touch duration, while the thin lines the individual ob- 
servers ones). 
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Figure 2.2.2.2. Orientation tuning of the effect of touch on binocular rivalry. 

The probability of maintaining the same visual stimulus during the whole touch period when 
visuo-haptic stimulation is parallel (Panel A, light grey symbols) and the probability of switch- 
ing perception when visuo-haptic stimulation is orthogonal (Panel B, dark grey symbols), 
compared to no-touch periods (open symbols) are plotted as a function of the mismatch in 
orientation between the visual and the haptic gratings (defined as haptic orientation - visu- 
al orientation). Both probabilities significantly differ from no-touch periods only when visual 
and haptic stimuli are perfectly matched in orientation (paired t-test, two-tailed, df=7, p<0.05). 


In the orientation discrimination experiment, similar to the main experiment, 
the visual stimuli were oriented at +45° and observers were asked at approximately 
regular intervals to explore the haptic stimulus with their right thumb. The orienta- 
tion of the haptic grating was randomly alternated each touch period and could be 
either clockwise (+45° relative to vertical) with an additional random offset of 0°, 
+7.5° or +15°, or counterclockwise (-45° relative to vertical) with an additional ran- 
dom offset of 0°, +7.5° or +15°. A diagram of the haptic orientation pairs is shown in 
Figure 2.2.2.3A. The visual stimuli alternated in simulated rivalry, as described 
above, and the observer’s task was to track the visual alternations by continuous 
keypress, thereby matching the conditions of the discrimination experiment and the 
original rivalry experiment as closely as possible. Only touch periods in which ob- 
servers accurately reported the visual orientation were considered for analysis. While 
tracking their visual perception, observers also judged during touch periods whether 
the haptic stimulus was tilted clockwise or counterclockwise relative to the visual 
stimulus (using a two-pedal floor switch). Because the visual stimulus was alternat- 
ing, in some touch periods the haptic grating was aligned (or nearly so) with the vis- 
ual grating (0°, +7.5° or +15°) while in other touch periods is was about 90° away 
(90° + 0°, +7.5° or +15°), as in the orthogonal visuo-haptic presentation condition 
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during the main binocular rivalry experiment. In either case, the response required 
was the same (the haptic grating was either clockwise or counterclockwise of the 
visual grating). 


Figure 2.2.2.3. Cross-modal orientation discrimination experiment: haptic stimuli and results. 
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Panel A shows the different haptic orientations used during the experiments. In the main ex- 
periment pairs of orthogonal haptic orientations were tested within every experimental block. 
In the cross-modal orientation discrimination experiment, clockwise and counterclockwise 
haptic orientations were tested in separate blocks, observers were required to judge to haptic 
orientation compared to the visual orientation during simulated binocular rivalry. Partici- 
pants were forced to report whether the haptic stimulus was oriented “further clockwise” or 
“further counterclockwise” compared to the congruent visual stimulus (either +45° or -45° 
relative to vertical). Panel B reports the proportion of “further clockwise responses” plotted as 
a function of the mismatch in orientation between the haptic and the visual stimuli. The small 
coloured symbols represent individual observers’ performances (every colour representing a 
different observer), the big transparent grey dots represent the average performance. Even 
though a trend of improving performance with increasing visuo-haptic orientation difference 
is present, observers’ performance did not significantly differ from the average performance 
for visuo-haptic stimuli matched in orientation neither for the 7.5 degrees difference (one 
sample t-test, df=4, t=2.405, p=0.074), nor for the 15 degrees difference for the 7.5 degrees 
difference (one sample t-test, df=4, t=1.95, p=0.123), meaning that they were not able to dis- 
criminate the difference in orientation between visual and haptic stimuli within 15° differ- 
ence. A (non significant) bias towards the “further clockwise” response was also observed 
when visuo-haptic stimuli were matched in orientation. 


The results of the visuo-haptic orientation discrimination experiment are re- 


ported in Figure 2.2.2.3B which plots the proportion of “further clockwise” respons- 
es as a function of angular difference. Threshold performance in this orientation dis- 
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crimination task is 0.50 correct, however for a visual-haptic orientation difference of 
0°, there was a bias to respond “clockwise”. That is observers judged the haptic stim- 
ulus matching the visual stimulus orientation as being oriented “further clockwise” 
-60% of the time. For this reason, we compared the +7.5° and +15° conditions 
shown in Figure 3B with average performance in the 0° condition, rather than the 
unbiased threshold of 50%. Performance for orientation differences of +7.5° and 
+15° did not significantly exceed performance in the 0° condition, implying that ob- 
servers were not able to discriminate the orientation difference between visual and 
haptic stimuli even when they differed by 15°. Importantly, discrimination perfor- 
mance in the “further counterclockwise” conditions (-7.5° and -15°), although not 
shown, did not differ significantly from the “further clockwise” conditions. These 
results indicate that participants were not aware of the visuo-haptic mismatch in ori- 
entation during the main experiment. That is, even though the effect of touch on ri- 
valry in the main experiment was confined to perfectly aligned visual and haptic 
stimuli (0° orientation difference), these discrimination data show that observers 
were not aware of the difference between 0° and 7.5°, or between or 0° and 15°. The 
lack of awareness of the visuo-haptic orientation difference therefore suggests that 
the strict orientation tuning found in the main experiment is not likely to be at- 
tributable to a categorical decision. 


2.2.2.3. Discussion 


We have shown that exploring a haptic grating while experiencing binocular ri- 
valry between orthogonally oriented visual gratings can substantially influence the 
alternation dynamics of binocular rivalry. Touching a grating that is congruent with 
the visual grating being perceived increases the likelihood that it will remain domi- 
nant. Conversely, touching a grating that is orthogonal to the visually perceived grat- 
ing increases the probability that perception will switch to the suppressed grating 
and therefore align the visual and haptic percepts (Figure 1D). Because the influence 
of the haptic grating acts not only on the perceptually dominant visual grating but 
also on the grating rendered invisible by binocular rivalry suppression (in contrast to 
attentional effects on rivalry, which generally influence only the dominant grating, 
Chong, Tadin, & Blake, 2005; Paffen & Alais, 2012), these results show that the in- 
fluence of touch on vision during binocular rivalry occurs outside of visual aware- 
ness and is therefore an automatic and compulsory interaction. 

One interesting aspect of the data is the relatively slow time course of the effect 
of touch in promoting dominance of the parallel visual stimulus. When we comput- 
ed the dynamics of the touch effect we found it takes at least 1 second to significantly 
bias rivalry and takes more than 2 seconds to peak after touch onset (Figure 1C). The 
slow time course is probably due to the role of adaptation and reciprocal inhibition 
in determining binocular rivalry dynamics. As was recently shown (Alais, Cass, 
O'Shea, & Blake, 2010), sensitivity to the two competing visual stimuli slowly chang- 
es during a single rivalry phase: initially sensitivity to the dominant stimulus is high 
and sensitivity to the suppressed stimulus is low. During a rivalry period, the sensi- 
tivity difference reduces due to adaptation of the dominant response and a corre- 
sponding release from inhibition of the suppressed response, reaching a near-zero 
difference just prior to a perceptual switch. Clearly, when the sensitivities to the 
competing stimuli are very similar (near the end of a rivalry period), the potential for 
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haptic input to bias visual competition would be greater. In our paradigm the touch 
stimulation was delivered at random moments relative to the rivalry process (near 
the middle of a rivalry phase, on average) and so we would expect the touch effect to 
increase over time as the current rivalry period extends and the relative strength of 
the visual stimuli converges. The time needed for touch to reach the peak effect 
therefore reflects the time-course of visual adaptation, and is not indicative of the 
time taken for haptic signals to feedback into early visual areas. 

One of the striking findings in this study is that the influence of touch on vision 
in binocular rivalry is orientation tuned, and very narrowly so (Figure 2). Indeed, 
our data show that the interaction requires matched visuo-haptic orientations, as a 
mismatch of 7.5° between visual and haptic orientations was sufficient to annul the 
effect of touch on binocular rivalry. We explored this narrow tuning further in a 
visuo-haptic orientation discrimination experiment. The results showed that observ- 
ers were not able to discriminate a 7.5° visuo-haptic orientation difference as being 
different from a 0° orientation difference (Figure 3). Nonetheless, although they 
were perceptually indistinguishable in orientation, our orientation tuning experi- 
ment established clearly that 7.5° was ineffective at biasing rivalry dynamics (Figure 
2). This pattern of orientation tuning shows a selectivity for orientation that is finer 
than conscious discrimination, suggesting that the unisensory signals do not need to 
be individually processed before being integrated. This is contrary to what would be 
expected in an optimal integration framework in which integration is thought not to 
be mandatory when integrating between the senses and which is thought to occur at 
a higher-level of processing after unisensory encoding (Hillis, Ernst, Banks, & Landy, 
2002). 

Fine orientation tuning is a characteristic of both early visual cortex (Hubel, 
Wiesel, & Stryker, 1977) and early somatosensory cortex (see Hsiao et al (Hsiao, 
Lane, & Fitzgerald, 2002) for a review) and links between visual and somatosensory 
systems has indeed been demonstrated (Fitzgerald, Lane, Thakur, & Hsiao, 2006; 
Zangaladze, Epstein, Grafton, & Sathian, 1999). In primary visual cortex, single-unit 
recordings show that cells typically exhibit a sharp orientation tuning with a band- 
width of approximately 15°, a bandwidth consistent with behavioural studies of ori- 
entation perception (Bradley, Skottun, Ohzawa, Sclar, & Freeman, 1987). Narrow 
orientation tuning of V1 cells has been reported in neurophysiological studies 
(Movshon & Blakemore, 1973; Phillips & Wilson, 1984). Given that the interaction 
between visual and haptic signals during binocular rivalry is tightly tuned for orien- 
tation, we conclude it is likely mediated by early visual neurons. This conclusion re- 
inforces the hypothesis first advanced by Lunghi et al (Lunghi, Binda, & Morrone, 
2010) that neural signals for touch and vision interact at the earliest stages of visual 
processing, probably V1. These authors based their conclusion on the fact that the 
visuo-haptic interaction they observed in binocular rivalry was tightly tuned for spa- 
tial frequency. Our finding of a tight orientation-tuned effect of touch on vision per- 
fectly complements their finding and adds converging evidence for an early visuo- 
haptic interaction. 

The orientation tuning of the visuo-haptic interaction that we observed is actual- 
ly narrower than that shown by visual neurons. One potential explanation of this we 
considered was that it was a case of ‘categorical perception’, a kind of non-linear 
perceptual response that can change abruptly around a boundary. Examples of cate- 
gorical visual perception have been found in face perception (Viviani, Binda, & Bor- 
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sato, 2007), familiar objects perception (Newell & Bulthoff, 2002) and colour percep- 
tion (Raskin, Maital, & Bornstein, 1983). To explain the orientation tuning of our 
effect, a categorical perceptual response could be envisaged which is thresholded to 
occur only when the visual and haptic gratings are iso-oriented and otherwise pro- 
duces a null response. We specifically tested this hypothesis in our discrimination 
experiment by testing whether subjects were aware of small differences between vis- 
ual and haptic orientation. The categorical perception hypothesis predicts they 
would be aware of the small differences because of its all-or-none response around 
0°. The fact that participants could not consciously discriminate the visuo-haptic 
mismatch in orientation within 15° difference (Figure 3) rules out the categorical 
perception hypothesis and instead suggests that the angular differences tested (0°, 
+7.5°, +15°) fell within a single orientation bandwidth and were therefore difficult to 
discriminate. The discrimination data also rule out a ‘response bias’ account of the 
peak interaction at 0° because any tendency to respond in a biased way to 0° would 
be evident for all orientations since they are perceptually indistinguishable. 

Another possible explanation of the narrow visuo-haptic orientation tuning is 
that it occurs as a result of optimal multisensory integration according to the maxi- 
mum likelihood estimation (MLE) model. In the MLE model (Ernst & Banks, 2002), 
multisensory signals are first encoded by unisensory processes and these estimates 
are then combined in a weighted linear sum. The weight for each sensory compo- 
nent is proportional to that component’s reliability, given by the inverse of its vari- 
ance. The model predicts that the combined estimate should have a lower variance 
than the unisensory estimates because of the following formula: o’vu = 
(0°v*024)/(02v+0%4), where 0°, is the variance of the visual estimate and o7y the vari- 
ance of the haptic estimate. The maximum improvement (lowering) in variance is by 
a factor of 2, which occurs when 07y = 0°y (Alais & Burr, 2004; Ernst & Banks, 2002; 
Ernst & Bulthoff, 2004). More relevant to orientation bandwidths, this means the 
visuo-haptic standard deviation (ovn) is reduced by a factor of V2. Could the MLE 
model therefore explain why the orientation tuning for the visuo-haptic interaction 
is narrower than is typically found in vision or in haptic perception? We think this is 
unlikely for several reasons. First, visual and haptic orientation tunings are not 
equal: the haptic tuning is broader than the visual one (Fitzgerald, Lane, Thakur, & 
Hsiao, 2006; Hsiao, Lane, & Fitzgerald, 2002) and hence the maximum reduction in 
ovu of V2 is not expected. Second, even making the assumption of equal bandwidths 
for visual and haptic orientation perception (which may be warranted because rival- 
ry suppression has been shown to broaden visual orientation tuning, Ling & Blake, 
2009), the sharp visuo-haptic orientation tuning we observed is much more than a 
factor of V2 narrower than the unisensory tunings and so is incompatible with the 
MLE model. 

A possible explanation for the sharp orientation tuning that we found would be 
to consider the haptic signal acting as a broadly orientation-tuned contrast pedestal. 
Cross-sensory pedestal effects between vision and touch have been recently found 
and are thought to reflect an early interaction between the two modalities. Arabza- 
deh et al (Arabzadeh, Clifford, & Harris, 2008) demonstrated that a visual flash pre- 
sented near the fingers during a simple haptic discrimination task was able to repro- 
duce the classical ‘dipper effect’ and improve near-threshold stimulus discriminabil- 
ity, as if the haptic signal had a direct input into the visual mechanism and provided 
the equivalent of a contrast pedestal. Similarly, in a speed discrimination task, Gori 
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et al (Gori, Mazzilli, Sandini, & Burr, 2011) showed cross-sensory facilitation be- 
tween vision and touch that resulted in a two-fold improvement of discrimination 
thresholds that was specific for matched visuo-haptic motion direction. Together, 
these two findings suggest that the haptic signal in our experiment, which is likely to 
be broadly tuned after being remapped into visual coordinates and fed back to early 
visual cortex, effectively provided a contrast pedestal for vision, thereby improving 
visual orientation discrimination and producing a very sharp tuning. 

Overall, our results add further converging evidence in support of the view that 
multisensory processing is present even in primary sensory cortices. Support for this 
view comes from a number of neurophysiological and anatomical studies showing 
unisensory inputs into other unisensory areas (for review see Driver & Noesselt, 
2008; Ghazanfar & Schroeder, 2006), as well as from sensory deprivation studies (re- 
viewed in Alais et al (Alais, Newell, & Mamassian, 2010) ) showing rapid recruit- 
ment of primary visual cortex for haptic processing observed in blind patients (Goy- 
al, Hansen, & Blakemore, 2006; Sadato et al., 1996). Indeed, even temporary loss of 
sight (e.g., 5 days) is sufficient to induce superior haptic performance in blindfolded 
individuals (Kauffman, Theoret, & Pascual-Leone, 2002; Merabet et al., 2008). Such 
rapid recruitment suggests somatosensory connections are not created ex novo after 
sensory deprivation but are already present in primary visual cortex in normal func- 
tioning and can be strengthened if needed (Pascual-Leone, Amedi, Fregni, & Me- 
rabet, 2005). In normal subject, these connections are likely to be weak compared to 
vision and easily masked by strong and reliable visual input. We propose that the in- 
herent signal ambiguity in binocular rivalry, in which two equally salient visual 
stimuli engage in a struggle for perceptual dominance, allows these relatively weak 
somatosensory inputs to exert a significant influence on early visual processing. 


2.2.3. Spatial proximity is crucial, active exploration is not. 


In the previous studies we used binocular rivalry to investigate cross-modal in- 
teractions between haptic and visual signals. We showed that a haptic signal can res- 
cue the visual stimulus congruent in orientation from binocular rivalry suppression, 
and that the interaction between visual and tactile signals during binocular rivalry is 
strictly tuned for matched visuo-haptic spatial frequencies and orientations. From 
these sets of evidence, we inferred that the haptic signal boosts the suppressed visual 
signal at the very early stages of visual processing, probably V1 or V2. This hypothe- 
sis is consistent with the existence of projections from the polysensory areas of the 
temporal lobe to V1 in monkeys (Falchier, Clavagnier, Barone, & Kennedy, 2002), 
with the evidence that the primary visual cortex of rats responds to tactile discrimi- 
nation of novel objects, and its activity correlates with tactile discrimination perfor- 
mance in an aperture discrimination task (Vasconcelos et al., 2011). Consistently, in 
normal sighted humans, BOLD studies show that primary visual cortex is activated 
during exploration of tactile dots (Merabet et al., 2007). V1 is recruited for tactile 
processing in blind patients (Sadato et al., 1996) and in normal sighted humans after 
prolonged blindfolding (Merabet et al., 2008). We therefore suggested that visual in- 
stability during binocular rivalry can reveal the somatosensory connections to pri- 
mary visual cortex that are normally masked by the strong visual input. 

We performed another set of experiments showing that the interaction between 
visual and tactile signals during binocular rivalry requires spatial overlap between 


68 


Claudia Lunghi 


visual and tactile stimuli, suggesting that already at the level of V1 - the putative site 
of the interaction - there must be a dynamic alignment between the two sensory 
spatial maps. In addition, we show that the effect does not require active exploratory 
action or an attentional allocation of the subject. However, the spatial frequency of 
the texture of the object must be closely matched. 


2.2.3.1. Materials and Methods 


Subjects 

Ten subjects (three males, average age 28+3.9 years), including one of the au- 
thors, participated in the experiment (subject AS took part only in two conditions); 
all had normal or corrected-to-normal vision, normal stereo acuity (Frisby Stere- 
otest, (Sasieni, 1978), with no strong eye dominance. Subjects gave informed con- 
sent. The experiments were carried out along the principles laid down in the declara- 
tion of Helsinki, and with approval of the relevant ethics committees. 

Apparatus and Stimuli 

The experiment took place in a dark and quiet room. Visual stimuli were created 
in MATLAB using PsychToolbox (Brainard, 1997), and displayed on a 24-inch mon- 
itor (Acer LCD GD245HQ), hung 37 cm over an opaque mirror. Observers viewed 
the reflection of the monitor at a distance of 20 cm from the mirror through ana- 
glyph red-blue goggles (right lens blue, left lens red). Responses were recorded 
through the computer keyboard. Visual stimuli were two oblique orthogonal red and 
blue gratings (orientation: +45°, size: 3°, SF 2 cpd or 3.5 cpd, contrast 30%), sur- 
rounded by a white smoothed circle, presented on a black uniform background in 
central vision. Peak luminance of the red grating was matched with the physical peak 
luminance of the blue one (2.13 cd/m’). The edges of the mirror were hidden by a 
black cloth, minimizing visual references. The tactile stimulus was a sinusoidal grat- 
ing (size: 3 cm, SF: 2c/cm) created with a 3D printer (Dimension Elite SST, Stratasys, 
Inc., Eden Prairie, MN). The stimulus was attached to a prop that could be moved 3 
cm back and forth under computer control with a linear trajectory at constant speed 
of 2.5 cm/sec. The motor and the stimulus prop were positioned under the mirror, 
with the tactile stimulus being at the same distance of the monitor (37 cm from the 
mirror), so that the visual stimulus was projected exactly on the tactile one. The ori- 
entation of the tactile stimulus could be changed by lever. Fig 2.2.3.1F illustrates the 
set-up. 

Task and Procedure 

Binocular Rivalry. Each observer participated in five 180-sec experimental ses- 
sions x five experimental conditions for a total time of 75 minutes, over different 
days. The three main tactile conditions were active exploration, passive touch, touch 
away. Active exploration and passive touch were tested also with non-matched 
visuo-tactile Spatial Frequencies (Visual SF: 3.5 cpd, Tactile SF: 2 c/cm). The order 
of the different conditions was randomized for every observer. 

After an acoustic signal (beep) the visual binocular rivalry stimuli appeared. Par- 
ticipants reported their visual perception by continuously pressing with the left hand 
one of two keys (up or down arrows) of the computer keyboard. They were instruct- 
ed to track color alternation; at each experimental session the orientation of the rival 
stimuli was swapped between the eyes. As assessed in training binocular rivalry ses- 
sions and in debriefing sessions, mixed percepts occurred for very brief periods, only 
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during perceptual transitions, and their frequency remained constant across condi- 
tions. 

In the active exploration condition, during the observing period, at approximate- 
ly regular intervals, observers were asked by the experimenter (verbal instruction) to 
explore the tactile stimulus with their right index finger performing a constant trans- 
lational movement, until the experimenter gave a stop signal (average touch period: 
2.5+0.15 sec). At each touch period the orientation of the tactile stimulus was ma- 
nipulated by the experimenter alternating the clockwise and counterclockwise orien- 
tations following preset random generated sequences (the orientation of the stimu- 
lus was unpredictable at each touch period). Touch periods timing and stimulus ori- 
entation were recorded by the experimenter by holding the proper mouse button. 

The touch away condition was identical to the active exploration except that the 
tactile stimulus was positioned 30 cm away (on the horizontal axis) from the loca- 
tion corresponding with the visual stimulus (center). At each touch period the ob- 
server accomplished a movement with the arm to reach the tactile stimulus (average 
touch period: 2.6+0.14 sec) and then went back to the resting position (centered on 
the visual stimulus). 

During the passive touch condition, the tactile stimulus was drifted 3 cm back 
and forth at 0.83 Hz. Observers wore insulating headphones to attenuate the sound 
of the motor, and kept their right index finger stationary on a foam rubber guide. 
The dynamics of the experiment were the same, except that touch periods were de- 
livered by the experimenter using a lever that elevated the tactile stimulus to touch 
the finger pad of the participant (average touch period: 2.6+0.15 sec, 2.8+0.14 sec for 
the mismatched spatial frequency condition). 

Tactile Priming. Two tactile priming durations (10 and 3 seconds) were tested in 
separate blocks comprising 30 consecutive trials. When cued by the acoustic signal 
(beep), together with the written instruction to “touch” observers started to actively 
explore the tactile stimulus with the right index finger. After 10 or 3 seconds (de- 
pending on the condition), a second beep sounded and the observers raised their 
finger from the stimulus, and the visual stimuli were displayed (red-blue gratings) 
for 3 seconds. The temporal delay from the beep signaling the end of touch and the 
presentation of the visual stimuli was set to 500 ms, but the actual separation be- 
tween the end of touch and the visual stimulus depended on the RT of the subjects. 
Using the computer keyboard, observers were asked to report their first percept (red 
or blue) by appropriate key pressing. A 2 seconds break was interleaved between tri- 
als, during which the experimenter changed the orientation of the tactile stimulus 
following preset random generated sequences. At each trial the orientation of the 
visual grating was swapped between the eyes. 


2.2.3.2. Results 


Interaction between visual and tactile signals during Binocular Rivalry 

We measured the effect of touch on binocular rivalry for active exploration 
and passive tactile stimulation conditions in separate blocks. In order to reduce 
the possibility of response bias, in all conditions observers were asked to track 
the color of the dominant stimulus (red or blue) and not the orientation (that 
was randomly associated with one of the two at every experimental block). In the 
conditions requiring active exploration of the tactile stimulus, observers were 
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asked at random intervals to explore the tactile stimulus with a horizontal 
movement of the right index finger. In the passive touch conditions the right in- 
dex finger of the observer was stationary on a prop and tactile stimulation was 
passively delivered with the tactile patch moving at a constant velocity of 2.5 
cm/s. The touch periods were brief (average 2.5s) and interleaved with vision- 
only stimulation; at each touch period the visual and the tactile stimuli could 
randomly be parallel or orthogonal to each other. 

We first computed the probability of switching, maintaining the same per- 
cept or switching more than once for the different conditions of visuo-tactile 
stimulation: parallel, orthogonal or, as a control condition, for period of visual 
only stimulation of comparable duration (Fig 2.2.3.1A-E). 

In the active-touch condition, when the visual and the tactile stimuli were 
parallel, the probability of maintaining the same percept for the whole touch pe- 
riod (Fig 2.2.3.1A, orange bars) increased by 32% compared with orthogonal 
visuo-tactile stimulation and by 20% compared with no-touch periods. When the 
visual and the tactile stimuli were orthogonal, the probability of switching (Fig 
2.2.3.1A, green bars) increased by 37% compared with parallel visuo-tactile 
stimulation and by 25% compared with no-touch periods. Consistent with our 
previous studies, active tactile stimulation influenced the dynamics of binocular 
rivalry, both by prolonging dominance and by curtailing suppression of the visu- 
al percept parallel to the tactile stimulus. Interestingly, the same result held for 
the passive-touch condition (Fig 2.2.3.1B): during parallel visuo-tactile stimula- 
tion the probability of maintaining the same percept for the whole touch period 
increased by 27% compared with orthogonal visuo-tactile stimulation and by 
17% compared with no-touch periods; similarly during orthogonal visuo-tactile 
stimulation the probability of switching increased by 32% compared with paral- 
lel visuo-tactile stimulation and by 21% compared with no-touch periods. In the 
passive condition the tactile signal was purely sensory: voluntary attention to the 
finger and active action were not involved (the finger was stationary in a resting 
position with tactile stimulation passively delivered by the experimenter, his/her 
only task was to report visual perception). 

Spatial proximity is known to be crucial for cross-sensory integration 
(Gepshtein, Burge, Ernst, & Banks, 2005). To test if it is also crucial in modulat- 
ing the visual rivalry, we repeated the experiment by placing the tactile stimulus 
30 cm away from the visual stimulus apparent location; at each touch period ob- 
servers were trained to reach the tactile stimulus, explore it with the right index 
finger and, after the stop signal, go back to the resting position aligned with the 
visual stimulus. We introduced the movement of the arm to re-set propriocep- 
tion, that is known to drift in the dark (Desmurget, Vindras, Grea, Viviani, & 
Grafton, 2000). When the visual and the tactile stimulus occupied different loca- 
tions (touch away condition) tactile stimulation did not influence the dynamics 
of binocular rivalry: none of the probabilities differs across visuo-tactile condi- 
tions (Fig 2.2.3.1C). Touch was ineffective also when the visual and the tactile 
stimulus were mismatched in spatial frequency at the same location (i.e. spatial 
frequency of the visual stimulus was 3.5 c/cm, while the tactile spatial frequency 
was 2 c/cm): maintaining, switching (once or more than once) are equally likely 
across visuo-tactile conditions both for active exploration of the tactile stimulus 
(Fig 2.2.3.1D) and passive tactile stimulation (Fig 2.2.3.1E). 
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A factorial 5x3 repeated measures ANOVA revealed a main effect of the type 
of tactile stimulation (parallel, orthogonal or none) both for the probability of 
maintaining the same percept for the whole touch period (F=17.704, p<0.001) 
and for the probability of switching once (F=8.639, p<0.01), but not for the 
probability of switching more than once (F=0.955, p=0.406). A main effect of 
experimental condition was found only for the probability of performing one 
switch during the touch period (F=3.085, p<0.05). A significant interaction of 
the two factors (experimental condition x type of touch) was found both for the 
probability of performing one switch during the touch period (F=23.458, 
p<0.001) and for the probability of maintaining the same percept during the 
whole touch period (F=6.806, p<0.001), but not for the probability of performing 
multiple switches s(F=1.975, p=0.065). The probability of maintaining and per- 
forming one switch significantly differed for the different type of tactile stimula- 
tions only for the active and passive touch conditions, as revealed by a paired 
two tailed t-test (Fig 2.2.3.1). 

We further analyzed the timecourse of the effect of touch on binocular rival- 
ry by computing the instantaneous probability of seeing the visual stimulus par- 
allel to the tactile stimulus as a function of time elapsed from the onset of touch 
(Fig 2.2.3.2A-E). AIl touch periods (independently from the type of visuo-tactile 
stimulation) were aligned at time zero, therefore the probability trace starts at 
chance level (with some random variability). In the active-touch condition (Fig 
2.2.3.2A), the average probability increased over time and reached statistical sig- 
nificance 0.7 s after the onset of touch, peaked at 1.7 s and then slowly decayed 
over time back to chance 1.5 seconds after the end of tactile stimulation. In the 
passive-touch condition, the probability trace reached statistical significance 1 s 
after the onset of touch, peaked at 2.3 s and reverted to chance 1 s after the end 
of tactile stimulation. The timecourse of the effect was slower than the active 
touch condition, while the decay of the effect was faster. Consistently with the 
average probabilities, when the visual and the tactile stimuli were in different lo- 
cations (Fig 2.2.3.2C) or were mismatched in spatial frequency (Fig 2.2.3.2D-E), 
the average probability trace was nearly flat and never reached a statistical signif- 
icant difference from chance level, indicating again that tactile stimulation did 
not interact with binocular rivalry under these conditions. Taken together these 
results show that spatial proximity and spatial frequency matching between visu- 
al and tactile stimuli are necessary conditions for their interaction during binoc- 
ular rivalry, while voluntary attention and action enhance the effect of touch on 
binocular rivalry, but are not necessary to attain the interaction. 
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Figure 2.2.3.1. Average Probabilities across Touch Conditions. 

The average probabilities of switching visual perception once (dark grey bar), maintaining the 
same visual percept (light grey bar) or switching more than once during a touch period (gray 
bar), depending on the type of visuo-tactile stimulation (parallel, orthogonal or no-touch peri- 
ods of comparable duration), plotted for the different experimental conditions (the error bars 
represent s.m.e.). Tactile stimulation influenced the dynamics of binocular rivarlry both during 
Active- and Passive-Touch conditions (Panel A-B): the probability of switching was significant- 
ly higher when the visual percept was orthogonal to the tactile stimulus, while the probability of 
maintaining the same visual percept for the whole touch period was significantly higher for par- 
allel visuo-tactile stimulation (paired t-test, n=10, two tailed, a = 0.025, *=p < 0.025, **=p<0.01, 
***=p < 0.001). When the visual and the tactile stimuli were in different locations, tactile stim- 
ulation had no effect on binocular rivalry (Panel C): none of the probabilities differed across 
visuo-tactile conditions. Tactile stimulation was also ineffective when the visual and the tactile 
stimuli were mismatched in Spatial Frequency, both for Active (Panel D) and Passive (Panel E) 
touch. Panel 1F shows the visual and the tactile stimuli. The visual stimuli were orthogonal red 
and blue gratings oriented at +45°, presented separately to the eyes through anaglyph red and 
blue goggles; the tactile stimulus was an engraved sinusoidal grating matched in Spatial Fre- 
quency with the visual stimuli (2c/cm). 
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Figure 2.2.3.2. Timecourse of the effect. 

The instantaneous probability of perceiving the visual stimulus parallel to the tactile stimulus 
is plotted as a function of time elapsed from the onset of touch. All touch-periods are aligned 
at time zero. The gray thin lines are the single subject traces, while the black thick line is the 
average trace. The vertical dashed line represent the average duration of a touch period. The 
two visuo-tactile conditions (parallel and orthogonal) are collapsed together. In the Active- 
Touch condition (Panel A) the probability trace is significantly biased towards the visual 
stimulus parallel to the tactile stimulus 0.724 s after the onset of touch, peaks at 1.65 s and 
then slowly decays to chance. In the Passive-Touch condition (Panel B) the probability trace 
is significantly higher than chance 0.96s after the onset of touch, peaks at 2.31 s and then 
slowly decays over time. When the visual and the tactile stimuli were in different locations 
(Panel C), or when the visual and the tactile stimuli were mismatched in Spatial Frequency 
(Panels D-E), the probability trace was flat and never significantly different from chance (T- 
test, two tailed, a = 0.025). Panel 2F is a cartoon of the experimental setup: an opaque mirror 
was placed half way between the monitor and the tactile stimulus (TS), so that the visual 
stimulus (VS) was projected onto the TS location. Responses were collected through the key- 
board. In the Passive-Touch condition, the TS was moved by a motor and tactile stimulation 
was delivered by the experimenter. 
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Tactile priming on Onset Rivalry 


The survival of the effect during passive tactile stimulation rules out the possibil- 
ity that voluntary action-related attention plays a critical role in mediating the inter- 
action. We ran another experiment in which one of the two rival images was primed 
by a tactile cue before the onset of rivalry. Object-based attention is known to drive 
the first coherent percept of binocular rivalry: if a visual cue is presented before the 
onset of rivalry, the first dominance phase will tend to be that of the cued stimulus 
(Chong & Blake, 2006; Hancock & Andrews, 2007; J. F. Mitchell, Stoner, & Reyn- 
olds, 2004). We therefore sought for a similar cross-modal effect by having observers 
explore the tactile stimulus for 10 s the in absence of visual stimulation, as they 
raised the finger, the visual stimuli were displayed for 4 seconds, and observers had 
to report their first coherent percept. The average delay between the cue and presen- 
tation of the visual stimuli was 500 ms, but the separation from touch was variable 
according to the promptness of observers in raising their finger when instructed. 


1.0 tactile cue duration = 10s 


Proportion of biased First Phases 


Figure 2.2.3.3. Tactile priming on Onset Rivalry. 

The proportion of first coherent percepts parallel to the tactile cue orientation is plotted for 
two conditions: in the first (Panel 3A) the duration of the tactile cue was 10 seconds, in the 
second condition (Panel 3B), the duration of the tactile cue was 3 seconds. The tactile cue did 
not prime the onset of rivalry: on average, the proportion of first dominance phases parallel 
to the tactile cue did not statistically differ from chance level (T-test, two tailed, a=0.025). The 
grey bars represent the single subjects’ data. 
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We found that the tactile cue did not bias the onset of rivalry: on average, the 
first coherent percept of binocular rivalry was at chance level relative to the tactile 
cue orientation (Fig 2.2.3.3A). In our paradigm 10 seconds of tactile stimulation did 
not affect the onset of rivalry in any direction, indicating that neither tactile priming 
or tactile adaptation occurred. We noticed that observers’ first dominant percept 
remained virtually unchanged: the visual stimulus presented to their dominant eye 
(irrelative of orientation) dominated most of the times and the tactile cue did not 
affect it. We repeated the experiment shortening the duration of the tactile cue to 3 
sec to match the duration of touch periods during binocular rivalry, but neverthe- 
less, tactile stimulation did not prime the onset of rivalry (Fig 2.2.3.3B). 


2.2.3.3. Discussion 


The results of this study confirm that tactile signals specifically interact with vi- 
sion during binocular rivalry promoting dominance of the parallel visual stimulus, 
both by prolonging its dominance and by shortening its suppression. This latter re- 
sult indicates that the interaction occurs outside of visual awareness, when the visual 
stimulus parallel to the tactile stimulus is perceptually suppressed during binocular 
rivalry. The relatively slow timecourse of the effect of touch on the dynamics of bin- 
ocular rivalry is in fact indicative of the time taken for the suppressed visual image to 
recover from the profound adaptation caused by binocular rivalry suppression 
(Alais, Cass, O'Shea, & Blake, 2010). The timing of the effect reflects the interaction 
of the tactile feedback to the visual areas with an intrinsically slow dynamics driven 
by adaptation and reciprocal inhibition between the monocular signals. 

The most interesting result of the present study is that the effect requires spatial 
proximity between the visual and the tactile stimulus, suggesting that the mecha- 
nisms mediating the effect have access to a spatiotopic spatial map that is invariant 
with the sensor position. The visual and tactile objects have to be matched also in 
texture, pointing to a functional role of the interaction. In addition, we demonstrate 
that an unambiguous tactile signal can resolve binocular rivalry even when tactile 
stimulation is passively delivered, showing that no voluntary or cross-modal atten- 
tion or action are necessary to mediate the facilitation. In the study by Lunghi et al, 
the visual and the haptic stimuli did not overlap (the visual stimulus was presented 
approximately 15 cm above the haptic stimulus location) and this could be consid- 
ered contradictory to the present results. However in that study, the haptic and visu- 
al stimuli were vertically aligned and laid on the same plane, moreover, both the arm 
and the hand of the observer were stationary for the whole testing block, with only 
the right thumb exploring the haptic grating. In this condition the subjects had the 
illusory perception of touching the visual stimulus probably caused by propriocep- 
tion adaptation. In the current study, when the visual and the tactile stimuli were 
misaligned, the observer had to actively reach the tactile stimulus performing a vol- 
untary action that reinforced the perception of misalignment between the visual and 
the tactile stimulus. Interestingly the effect reported in the current study is slightly 
stronger than in the previous study, reinforcing the fact that spatial alignment is im- 
portant. 

The unconscious, automatic nature of the effect of touch on binocular rivalry 
differentiates our results from the vast evidence that optimal integration between 
cross-sensory signals is not mandatory (Hillis, Ernst, Banks, & Landy, 2002), and 
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from previous studies on cross-sensory interactions with ambiguous visual stimuli. 
Auditory looming signals, when attended, can prolong the dominance phase of the 
visual stimulus of congruent temporal frequency during binocular rivalry, but do not 
curtail its suppression (van Ee, van Boxtel, Parker, & Alais, 2009). The same is true 
for auditory stimuli semantically congruent with one of the visual rivaling stimuli (a 
bird and a car respectively: Chen, Yeh, & Spence, 2012) and for an auditory stimulus 
amplitude modulated at a rate matching one of the rival stimuli spatial frequency 
(Guzman-Martinez, Ortega, Grabowecky, Mossbridge, & Suzuki, 2012b). 

Previous studies have shown that touch can help disambiguate visual perception: 
touching a rotating globe prolongs dominance of the congruent rotation direction in 
the kinetic depth effect (Blake, Sobel, & James, 2004), kinestethic information can 
solve the aperture problem when observers actively move a cube aligned with the 
visual stimulus in one of the two component directions of the visual stimulus (Hu & 
Knill, 2010) and touch merges with vision to improve slant discrimination (Ernst, 
Banks, & Bulthoff, 2000). Fusion between visual and tactile signals during these 
tasks, however, is not mandatory and requires conscious perception of each of the 
unisensory signals. This evidence is coherent with the view that cross-modal signals 
are integrated at higher level associative brain areas only after being analyzed by the 
unisensory cortices. Our results challenge this view and are in line with a new con- 
cept that considers the whole brain as essentially multisensory (Ghazanfar & 
Schroeder, 2006). 

Cross-modal attention modulates the activity of early visual cortices (Macaluso, 
Frith, & Driver, 2000) and could potentially mediate the effect that we observed. We 
therefore tested whether a cross-modal cue could prime the onset of rivalry. While 
exogenous attention has a minor effect on sustained binocular rivalry, there is evi- 
dence that an object-based attentional cue presented before the onset of rivalry can 
substantially bias the first coherent percept in favor of the cued stimulus (Chong & 
Blake, 2006; Hancock & Andrews, 2007; J. F. Mitchell, Stoner, & Reynolds, 2004). 
Onset rivalry, in fact, is thought to have different properties from “late” rivalry: it is 
quite stable and is susceptible to different biases (Stanley, Forte, Cavanagh, & Carter, 
2012). The intermittent presentation of the rivalry stimuli “stabilized” binocular ri- 
valry: an effect first described by (Leopold, Wilke, Maier, & Logothetis, 2002) that 
has been attributed to perceptual memory for ambiguous figures. When primed by 
touch the observers of the present study reported the same percept over several trials 
and touch could not disrupt this stabilization. The ineffectiveness of a tactile cue to 
prime the onset of rivalry suggests that simultaneous presentation of visual and tac- 
tile stimuli is a crucial factor for the interaction to occur, and rules out the possibility 
that the interaction between vision and touch during binocular rivalry could be en- 
tirely explained by activating attentive mechanisms with cross-modal cues. This con- 
trol additionally rules out the possibility of response bias. 

It has been demonstrated that voluntary action interferes with a visual signal 
suppressed by binocular rivalry (Maruya, Yang, & Blake, 2007), the persistence of 
the effect in the passive touch condition, therefore indicates that the nature of the 
visuo-tactile interaction is exclusively sensory. Importantly, that the effect of touch 
on rivalry depends on congruency of visuo-tactile spatial frequency for passive tac- 
tile stimulation further confirms that the somatosensory signal alone is sufficient to 
boost the suppressed visual signal at the very first stages of visual analysis: only neu- 
rons of the primary visual cortex in fact show spatial frequency tuning as narrow as 
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less than one octave (Maffei & Fiorentini, 1973), necessary to explain the selectivity 
of our effect. V1 therefore seems the prime candidate neural locus for the visuo- 
tactile interaction that we report. The finding that spatial proximity is necessary to 
fuse the visual and the tactile signals extends the specificity of the interaction: the 
visual and the tactile signals must be perceived as arising from the same object to be 
integrated. However, this also poses a problem. V1 neurons code visual space in ret- 
inotopic coordinates, while tactile information is initially coded in somatotopic 
space: only later at parietal levels there is evidence of mapping in visual retinotopic 
space (Andersen, Essick, & Siegel, 1985). To explain both the spatial frequency selec- 
tivity and the spatial alignment of the two sensory maps we have to hypothesize that 
the interaction take place in V1, but it is mediated by a tactile signal represented in 
visual retinal coordinates. There is no evidence of direct projections from S1 to V1, 
but there is evidence of projections from STP (Falchier, Clavagnier, Barone, & Ken- 
nedy, 2002) where somatosensory information may be mapped in retinotopic coor- 
dinates (Andersen, Essick, & Siegel, 1985). These projections back to V1 may medi- 
ate the effect observed here. 

Taken together, the results presented here suggest that the interaction between 
visual and tactile signals during binocular rivalry takes place at the very early stages 
of visual processing, probably V1. We propose that the somatosensory projections to 
the primary visual cortex, normally concealed by the strong and unambiguous visual 
signal are revealed by the temporary blindness caused by binocular rivalry suppres- 
sion, and are shown in this study to be “retinotopically” coded. These results bring 
new evidence in favor of a novel role of primary sensory cortices, which have been 
thought to process only unisensory information, as well as highlighting the debate 
about neural plasticity following sensory loss. 


2.2.4. Summary and Conclusions 


We have demonstrated that touch can disambiguate visual perception during 
binocular rivalry favouring dominance of the congruent visual stimulus both by pro- 
longing its dominance duration and by shortening its suppression. That the tactile 
signal can rescue the congruent visual signal from binocular rivalry suppression re- 
verting it to consciousness indicates that the interaction between vision and touch 
during binocular rivalry is compulsory, for it occurs outside of visual awareness. By 
varying the spatial frequency and orientation of the tactile stimuli we have also 
shown that the interaction between vision and touch during rivalry is specific, with a 
very tight tuning for orientation and spatial frequency: the visual and the tactile 
stimulus need to be perfectly matched both in spatial frequency and in orientation in 
order to observe the interaction. We have further demonstrated that voluntary atten- 
tion and action are not necessary conditions for the interaction to occur: passive tac- 
tile stimulation is as effective as active haptic stimulation in biasing binocular rivalry. 
We have instead found that spatial and temporal alignment are crucial factors for 
the interaction, for placing the haptic stimulus away from the visual stimulus loca- 
tion or delivering tactile stimulation before the onset of visual stimulation annulled 
the effect of touch on the dynamics of binocular rivalry. 

Taken together, the results listed above strongly point to an early interaction be- 
tween visual and tactile signals during binocular rivalry: 
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e binocular rivalry suppression occurs early in the visual system, neural activity 
associated with the suppressed image is not traceable outside V1 or V2. In or- 
der to boost the suppressed visual signal, the tactile signal must reach these ear- 
ly visual cortices; 

e touch interacts with vision outside of visual awareness, indicating that the in- 
teraction can occur before processing of unisensory information; 

e neurons showing a tuning for orientation and spatial frequency as narrow as 
that we found for the interaction are typically found in primary visual cortex; 

e spatial and temporal alignment are benchmarks of early cross-modal integra- 
tion. 


We propose that the intrinsic ambiguity of binocular rivalry and the temporary 
blindness provoked by binocular rivalry suppression are able to reveal the soma- 
tosensory connections with early visual cortices, connections that are normally con- 
cealed by the strong and reliable visual input. Our results expand the evidence in fa- 
vour of the involvement of early sensory cortices in multisensory processing, provid- 
ing psychophysical evidence in favour of an early interaction between vision and 
touch. In conclusion, our findings support the view that the recruitment of primary 
visual cortex for tactile processing observed following loss of vision occurs by rein- 
forcing pre-existing connections, rather than by creating them ex-novo. 
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Chapter 3 
Plasticity of adult human visual cortex revealed by binocular rivalry. 


3.1. Introduction 


Neuroplasticity refers to the capability of the nervous system to be modified by ex- 
perience. The term plastic in fact derives from the Greek word plastòs that means 
molded. Such experience-induced neural changes may be driven by environmental 
pressure, functional significance, behavioural relevance and neural activity and can oc- 
cur through structural or functional alteration both at the macroscopic (i.e. neural 
connections) and microscopic (i.e. synapses) levels. Neuroplasticity is maximum early 
in life and is crucial for the development of the organism, for it ensures adaptability to 
a continuously changing external environment providing an evolutionary advantage 
for the organism. Besides being an adaptive phenomenon, sometimes neuroplasticity 
can be “negative”, or maladaptive, with experience-driven modifications resulting in 
pathologies. During adulthood, neuroplasticity is thought to be reduced, especially for 
low level sensory processing, but it is preserved for higher level properties, for plastici- 
ty is the process underpinning learning and memory. Because of its leading role in di- 
verse neural processes, neuroplasticity is thought to be an intrinsic property of the 
nervous system (Pascual-Leone, Amedi, Fregni, & Merabet, 2005). 

In the next paragraphs, we will focus on plasticity of the visual system, reviewing 
experimental evidence regarding the plastic potential of early visual cortex both dur- 
ing development and adulthood. 


3.1.1. Juvenile plasticity of the visual system: the critical period. 


The developing mammalian visual system is extremely plastic: early in life, 
within a particular temporal window called critical period, the visual cortex is liter- 
arily shaped by experience, for it is immature at birth. After the closure of the crit- 
ical period, visual plasticity is thought to be severely reduced or even absent, visual 
cortex being hard-wired during adulthood. Critical periods have been found in all 
species for different sensory modalities and within a sensory modality for different 
functions (Berardi, Pizzorusso, & Maffei, 2000). The duration of the critical period 
varies across species and depends on the feature tested. Since the critical period is 
a well defined part of the animal life during which neural connections are shaped 
by experience, its duration is related to the animal lifespan: the longer the lifespan, 
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the longer the critical period. While the extraordinary plasticity of the sensory 
brain is crucial during development, allowing the organism to adapt to the exter- 
nal world escaping the limitations of its gene pool, an excessive duration of the 
sensitive period would expose the organism to the possibility of maladaptive neu- 
ral modifications driven by an abnormal visual input. Figure 3.1.1. shows the cor- 
relation between the duration of the critical period for ocular dominance (one of 
the most studied features of the visual cortex that we will detail in the next para- 
graphs) and the lifespan of different mammal species: mouse (Huang et al., 1999), 
rat (Fagiolini, Pizzorusso, Berardi, Domenici, & Maffei, 1994), ferret (Issa, 
Trachtenberg, Chapman, Zahs, & Stryker, 1999), cat (Olson & Freeman, 1980), 
monkey (Harwerth, Smith, Duncan, Crawford, & von Noorden, 1986) and human 
(Banks, Aslin, & Letson, 1975). Finally, a correlation also exists between critical 
period duration and brain complexity (and related brain weight): the more com- 
plex the brain, the longer the critical period. 
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Figure 3.1.1. Ocular Dominance critical period duration and mammal lifespan. Adapted from 
(Berardi, Pizzorusso, & Maffei, 2000). 

The duration of the critical period for ocular dominance has been studied in different mam- 
mal species: mouse, rat, ferret, cat, monkey and human. A linear correlation (r=0.98, 
SD=0.14,p<0.001) exists between the duration of the critical period and the animal lifespan. 


3.1.1.1. Ocular Dominance plasticity during the critical period. 


In the young animal, cortical areas are not yet fully specialized for a particular 
task. Neuroplasticity is the mechanism through which cortical differentiation is re- 
fined during the first period of life, tuning cortical activity according to the envi- 
ronmental pressures. The initial sensitive period during which neural connections 
are shaped by experience is a protection for the young animal, for brain injury can 
be compensated by other brain areas taking charge of the damaged functionality 
(Stiles, 2000). On the other hand, as anticipated in the previous paragraph, if for 
some reason the early experience is inadequate, plasticity will mold brain structure 
to the abnormal input, causing permanent deficit to the developing organism. One 
of the most studied cases of this maladaptive form of plasticity is the development of 
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ocular dominance columns and binocularity in the primary visual cortex, that can be 
severely damaged by inadequate visual input, resulting in amblyopia if vision in one 
eye is impaired during development. 
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Figure 3.1.2. Effect of monocular deprivation on ocular dominance. Adapted from (Wiesel & 
Hubel, 1965a) 

(A) Organization of ocular dominance in the normal cat, neurons in primary visual cortex are 
classified according to their monocular preference. Group 1 cells respond only to the contra- 
lateral eye, while group 7 cells respond only to the ipsilateral eye. Group 4 neurons respond 
equally to both eyes, while group 5 and 6 respond to both eyes, but more to the ipsilateral eye 
and group 2 and 3 respond to both eyes, but more to the contralateral eye. Adapted from 
(Hubel & Wiesel, 1962) (B) Following monocular deprivation during the critical period, all 
cortical cells respond to the open eye, while responses to the deprived eye are abolished, and 
as a consequence, none of the neurons shows binocular responses. Adapted from (Wiesel & 
Hubel, 1963b) (C) Following binocular deprivation during the critical period, ocular domi- 
nance organization of primary visual cortex is almost entirely preserved, showing that com- 
petition between monocular inputs is critical for cortical organization. 


In the 1960s, the Nobel Prize’s David Hubel and Torsten Wiesel studied the de- 
velopment and plasticity of ocular dominance columns in the primary visual cortex 
of the cat (Hubel & Wiesel, 1962, , 1963, , 1964; Wiesel & Hubel, 1963a, , 1963b, , 
1965b). In their pioneering work, Hubel and Wiesel demonstrated how even a short 
epoch of monocular deprivation during the critical period caused a rearrangement 
of thalamo-cortical connections in the kitten brain, shifting ocular dominance in fa- 
vour of the open eye. In the normal primary visual cortex, the majority (4/5) of neu- 
rons are binocular, that is they respond to inputs presented to both eyes, while only 
1/5 of neuron shows monocular responses. Binocular neurons can be in turn differ- 
entiated according to their monocular preference, even though they respond to bin- 
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ocular inputs, in fact, some binocular neurons show a preference for one or the other 
eye (Hubel & Wiesel, 1962). Hubel and Wiesel (1962) classified neurons in the pri- 
mary visual cortex of the adult cat according to their monocular preference, individ- 
uating seven different classes of neurons, where neurons of group 1 are completely 
driven by the contralateral eye and neurons of group 7 are totally driven by the ipsi- 
lateral eye (Figure 3.1.2.A). Neurons falling in the intermediate groups respond to 
input from both eyes, but show a preference for either the contralateral or the ipsi- 
lateral eye, with only neurons of group 4 responding equally to stimuli presented to 
the two eyes (Hubel & Wiesel, 1962). In a subsequent study, ocular dominance or- 
ganization of young kittens has been showed to be very similar to that of the adult 
(Hubel & Wiesel, 1963). 

Monocular deprivation has been classically used as a probe for ocular domi- 
nance plasticity: if, during the critical period, one eye is deprived of vision either by 
lid suture or by the application of an opaque contact lens, the deprived eye loses the 
ability of driving cortical activity, and all cortical neurons respond to the normal eye, 
becoming therefore monocular (Figure 3.1.2.B). The normal eye takes over the con- 
nections of the deprived eye, which results in a shrinkage of ocular dominance col- 
umns of the deprived eye and a relative enlargement of those of the open eye. As a 
result, the kitten becomes amblyopic, with defective vision (loss of visual acuity) in 
the eye that had been deprived as well as a loss of binocular vision, with consequent 
behavioural disadvantages (Wiesel & Hubel, 1963a; 1963b). The deafferentation of 
primary visual cortex observed following monocular deprivation is due to the weak- 
ening, and eventually deletion, of thalamo-cortical connections from the deprived 
eye. 

This dramatic shift of ocular dominance following monocular deprivation, can 
be recovered by patching the normal eye within the critical period, this reverse 
patching procedure, completely restores normal cortical organization in young kit- 
tens (Blakemore & Van Sluyters, 1974), while, after the closure of the critical period, 
reverse patching has only a very limited effect (Wiesel & Hubel, 1965b), suggesting 
that experience-dependent plasticity is limited during adulthood. Competition be- 
tween monocular inputs seems to be a crucial factor driving the plastic changes ob- 
served during the critical period. Binocular deprivation, in fact has been demon- 
strated to have less severe consequences on primary visual cortex organization (Fig- 
ure 3.1.2.C), with ocular dominance of binocularly deprived kittens being similar to 
normal (Wiesel & Hubel, 1965a). 

Hubel and Wiesel (1970) thoroughly investigated the timecourse of ocular dom- 
inance plasticity in kittens, and found that susceptibility of primary visual cortex to 
the effect of monocular deprivation starts at around 3-4 weeks of age, it then peaks at 
around 6-8 weeks, when even a few days of monocular deprivation produce a large 
shift of ocular dominance in favour of the normal eye. After the peak sensitivity, ex- 
perience-dependent plasticity slowly declines until 3 months of age, when the critical 
period ends. Importantly, the onset of the critical period can be delayed by rearing 
animals in the darkness, dark rearing results in slowing the timecourse of the critical 
period, showing that visual experience is necessary to trigger neuroplasticity (Mow- 
er, 1991). After the closure of the critical period, even prolonged periods (up to one 
year) of monocular deprivation do not affect ocular dominance organization (Hubel 
& Wiesel, 1970). We will discuss residual adult plasticity later in a dedicated para- 
graph. 
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3.1.1.2. Critical Period Regulation. 


As detailed in the previous paragraph, the critical period for ocular dominance 
plasticity follows a well-defined timecourse that is definite for every specie. What are 
the factors determining the onset and the closure of the critical period? 

The current understanding of the neural and molecular mechanisms controlling 
critical periods has been obtained from experiments on the primary visual cortex of 
mice, for using transgenic mice is a powerful method to investigate the molecular 
and cellular components underlying critical periods regulation. Another advantage 
of studying ocular dominance plasticity in mice is that their critical period is much 
shorter than that of superior mammals (see paragraph 3.1.1.), occurring between 
postnatal day 28 and postnatal day 32 (Gordon & Stryker, 1996), so that even few 
days of monocular deprivation are sufficient to generate ocular dominance re- 
organization, speeding up the times of research. Since the beginning of the use of 
transgenic mice for the study of ocular dominance plasticity, several important dis- 
coveries have been made about critical period regulation. Here we will focus on the 
role of balance between intracortical inhibition and excitation, that seems to be one 
of the most important factors underpinning the timecourse of neuroplasticity. 

The first evidence revealing that the balance between excitation and inhibition is 
important for the modulation of ocular dominance plasticity was the discovery that 
the development of GABAergic inhibition was necessary to trigger the onset of the 
sensitive period. Hensch et al (1998) demonstrated that ocular dominance plasticity 
could not be induced in transgenic mice in which the GABA-synthesizing enzyme 
glutamic decarboxylase (GAD665) had been deleted, reducing therefore the synthe- 
sis of GABA. Interestingly, in these mice, independently of their age (so even during 
adulthood) the critical period for ocular dominance could not be restored until they 
were treated with benzodiazepines that are GABAA receptors agonists (Hensch et 
al., 1998). Similarly, in normal mice, the treatment with benzodiazepines can antici- 
pate the onset of the critical period, supporting the hypothesis that an inhibitory 
threshold must be reached to initiate the sensitive period (Fagiolini & Hensch, 
2000). Importantly, the precocious critical period produced by the injection of ben- 
zodiazepines is not observed in mutant mice whose GABAA receptors alphal subu- 
nits have been rendered insensitive to benzodiazepines, suggesting a precise role of 
inhibitory interneurons (fast-spiking Parvalbumim expressing basket cells) in medi- 
ating critical period onset (Fagiolini et al., 2004). 

In line with these results, transgenic mice over-expressing the neural growth fac- 
tor BDNF show an earlier development of inhibitory boutons compared with wild- 
types, that is accompanied by an anticipation of the critical period onset as well as by 
an acceleration of high visual acuity development (Hanover, Huang, Tonegawa, & 
Stryker, 1999; Huang et al., 1999). Interestingly, when these mice were reared in the 
dark, the usual delay of critical period onset provoked by dark rearing (Fagiolini, 
Pizzorusso, Berardi, Domenici, & Maffei, 1994) was not observed, and the animals 
presented high acuity vision (Gianfranceschi et al., 2003). Similar effects have been 
found for the over-expression of the insulin growth factor -1 (IGF-1) (Ciucci et al., 
2007), that is known to stimulate the production of BDNF (Carro, Nunez, Busiguina, 
& Torres-Aleman, 2000), that in turns regulates the development of intracortical in- 
hibition. 
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The evidence listed above clearly demonstrates that the onset of the critical peri- 
od depends on the development of GABAergic inhibition. Recent evidence suggests 
that the continued maturation of inhibitory innervations is also one of the factors 
contributing to the closure of the critical period for ocular dominance plasticity. One 
way of investigating the mechanism underlying the closure of the critical period is 
studying which factors can re-open a critical period in adult animals. Several evi- 
dence have shown that changing the balance between inhibition and excitation in 
primary visual cortex can restore plasticity in adult visual cortex. For example, ocu- 
lar dominance plasticity has been induced in adult rats housed in an enriched envi- 
ronment (Sale et al., 2007), this renewed neuroplasticity was associated with a de- 
crease in GABAergic inhibition, through a reduction of the vescicular GABA trans- 
porter (VGAT). The importance of the reduction of intracortical inhibition in driv- 
ing the reopening of the critical period in adult animals was confirmed in the same 
study by showing that the treatment with benzodiazepines blocked the visual cortex 
plastic potential in the animals exposed to the enriched environment (Sale et al., 
2007). 
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Figure 3.1.3. Inhibition and Plasticity during the critical period. Adapted from (Heimel, van 
Versendaal, & Levelt). 
Abbreviations: EE is enriched environment, DT is dark treatment, PNN is perioneural net. 


The treatment with benzodiazepine has been also shown to be effective in block- 
ing adult ocular dominance plasticity induced by the administration of the selective 
serotonin reuptake inhibitor (SSRI) Fluoxetine (Maya Vetencourt et al., 2008), 
showing that both increasing excitation and decreasing inhibition are important for 
neuroplasticity. A direct evidence for decreased inhibition being one of the factors 
improving adult cortical plasticity was provided by Harauzov et al (2010): when the 
levels of intracortical inhibition were pharmacologically lowered by treating adult 
rats with a GABA antagonist (picrotoxin) or with an inhibitor of GABA synthesis 
(MPA) during monocular deprivation, ocular dominance plasticity was increased in 
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adult animals (Harauzov et al., 2010). Moreover, by manipulating the Lynx1 protein 
in adult mice, Morishita et al (2010) showed that cholinergic innervations is also im- 
portant for ocular dominance plasticity. When Lynx1 was removed in transgenic 
mice, enhancing nicotinic receptors signaling (increasing the cortical levels of cho- 
linergic excitation), neuroplasticity was reactivated during adulthood, confirming 
the importance of the balance between excitatory and inhibitory circuits in regulat- 
ing critical periods (Morishita, Miwa, Heintz, & Hensch, 2010). Figure 3.1.3. illus- 
trates in a diagram the relationship between inhibition and ocular dominance plas- 
ticity. 

Another factor contributing to the closure of the critical period for ocular domi- 
nance plasticity is the limitation of structural plasticity. For example, the epigenetic 
regulation of gene expression (ERG-CREB) has been shown to be important for neu- 
roplasticity during the critical period, for interfering pharmacologically with this 
process re-opens a critical period for ocular dominance in adult animals (Putignano 
et al., 2007). Furthermore, removing Nogo-66 receptors results in continuation of 
the critical period throughout adulthood, the factor Nogo being a myelin-derived 
factor inhibiting axon development (McGee, Yang, Fischer, Daw, & Strittmatter, 
2005). Finally, when the extracellular matrix (that blocks experience-dependent plas- 
ticity in the adult cortex by inhibiting axon and spine growth) is chemically dis- 
solved in V1, adult animals become sensitive to monocular deprivation (Pizzorusso 
et al., 2002). This latter finding well illustrates how inhibitory innervations and 
structural plasticity are closely linked in determining neuroplasticity: a thick perio- 
neuronal net in the extracellular matrix (forming therefore a thicker barrier for plas- 
tic changes) is in fact observed around the Parvalbumim expressing basket cells 
(Luth, Fischer, & Celio, 1992) that, as detailed above, are inhibitory interneurons 
crucial for ocular dominance plasticity. Consistently with this evidence, in transgen- 
ic mice lacking of the protein Ctrll (that regulates the formation of the perioneu- 
ronal nets), ocular dominance plasticity is preserved during adulthood (Carulli et al., 
2010). 


3.1.2. Visual cortical plasticity in the adult. 


In the previous paragraph we have detailed how ocular dominance plasticity is 
defined and regulated during the critical period. Until recently, ocular dominance 
plasticity was thought to be restricted during the critical period, adult visual cortical 
circuits being hard-wired. However, during the last decade growing evidence has 
demonstrated a residual plastic potential of adult visual cortex. This has become pos- 
sible because of the use of mice to study the mechanisms underlying ocular domi- 
nance plasticity, as mice, compared with other species, show a higher degree of adult 
plasticity (Pizzorusso et al., 2002; Sato & Stryker, 2008). 

Compared to juvenile plasticity, longer periods of monocular deprivation are re- 
quired to observe a shift of ocular dominance in adult animals (Blakemore, Garey, & 
Vital-Durand, 1978; Issa, Trachtenberg, Chapman, Zahs, & Stryker, 1999; Pham et 
al., 2004; Sawtell et al., 2003; Tagawa, Kanold, Majdan, & Shatz, 2005), and the shift 
of ocular dominance after prolonged deprivation is still smaller than that observed 
for shorter deprivation periods during the sensitive period. Moreover, while in 
young animals the ocular dominance shift occurs predominantly through a rapid 
depression of deprived-eye responses and in a subsequent reinforcement of the open 
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eye responses (Frenkel & Bear, 2004; Mrsic-Flogel et al., 2007), in adult mice, the 
major change observed following monocular deprivation is an increase of respon- 
siveness of the open eye (Hofer, Mrsic-Flogel, Bonhoeffer, & Hubener, 2006; Sato & 
Stryker, 2008; Sawtell et al., 2003). 

Even if a residual ocular dominance plasticity is found in adult visual cortex, it is 
quantitatively and qualitatively different from that observed during the critical peri- 
od, moreover, this enduring plasticity can only be observed in mice, limiting the 
generalization of the finding to other species. Although ocular dominance plasticity 
may be particularly constrained within the critical period, adult visual cortical plas- 
ticity has been reported for other properties. In the next paragraph we will review 
the most important discoveries regarding visual cortical plasticity during adulthood 
following retinal lesions, during the recovery from blindness and following pro- 
longed visual adaptation. 


3.1.2.1. Adult visual cortical plasticity following retinal lesions. 


Reorganization of cortical topography has been observed in the adult visual cor- 
tex of monkeys and mice following retinal lesion. The cortical changes observed in 
the lesion projection zone (LPZ, that is the retinotopic region in primary visual cor- 
tex corresponding to the representation of the damaged part of the retina), after bin- 
ocular lesions entail changes in receptive field properties, in molecular mechanisms 
and in neural circuitry (for review see Gilbert & Li, 2012). After an initial depression, 
the neurons within the LPZ zone regain responsiveness to visual stimulation, as a 
result of wide structural reorganization occurring in primary visual cortex, with neu- 
rons within the LPZ being driven by neurons in cortical regions surrounding the ar- 
ea representing the retinal lesion (Gilbert, Hirsch, & Wiesel, 1990; Kapadia, Ito, Gil- 
bert, & Westheimer, 1995). This reorganization is thought to occur through the 
strengthening of horizontal connections that in the normal brain are thought to 
spread information across the visual map, allowing contour integration in primary 
visual cortex (Gilbert & Li, 2012). 

Interestingly, the first change observed in adult rat cortex following the retinal 
lesion is a reduction of GABAergic inhibition (Mittmann et al., 1994), that has been 
shown to occur through a massive structural synaptic reorganization of GABAergic 
interneurons (Keck et al., 2008; Keck et al., 2011). This result points again to a lead- 
ing role of the balance between intracortical inhibition and excitation in regulating 
visual cortical plasticity. A decrease of inhibition might be necessary for initiating 
experience-dependent plasticity in the adult visual cortex. 

A reorganization of retinotopic maps in primary visual cortex, similar to that 
found for animals after retinal lesions, has been shown in humans with fMRI tech- 
niques. For example, Baker et al (2005) tested BOLD responses in two patients 
whose retinal fovea was damaged because of macular degeneration (Figure 3.1.4A). 
When measuring BOLD in the occipital cortex of these patients, that have a large 
binocular central retinal lesion, Baker et al (2005) found a strong activation in the 
foveal confluence for peripheral visual stimuli (Baker, Peli, Knouf, & Kanwisher, 
2005). This result clearly indicates a large-scale reorganization of visual processing, 
with a big portion (20 cm’) of deafferented visual cortex responding to visual stimuli 
presented in the periphery of the visual field, a reorganization that is probably due to 
the horizontal connections spreading information from areas receiving sensory in- 
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put (those representing peripheral stimuli), to areas deprived because of macular de- 
generation (the foveal confluence, that, because of the cortical magnification factor 
for human foveal retina (Sereno et al., 1995), is a large portion of the visual cortex). 
Interestingly, in macular degeneration patients, the portion of visual cortex activated 
by peripheral visual stimuli was silent for central visual stimulation, while in normal 
control subjects the opposite held (Figure 3.1.4A): the foveal confluence was activat- 
ed during central visual stimulation and silent during peripheral visual stimulation 
(Baker, Peli, Knouf, & Kanwisher, 2005). 
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Figure 3.1.4. Topographic reorganization of adult human visual cortex following retinal le- 
sion and stroke. 

(A) Adapted from (C. I. Baker, Peli, Knouf, & Kanwisher, 2005). BOLD activation of two 
macular degeneration patients (MD1 and MD2) visual cortex to visual stimuli presented in 
the periphery of the visual field, compared with a group of control normally sighted subjects. 
In controls, peripheral stimulation did not activate the large portion of the visual cortex that 
responds to foveal stimulation (foveal confluence), that was instead activated in macular de- 
generation patients. 

(B) Adapted from (Dilks, Serences, Rosenau, Yantis, & McCloskey, 2007). {MRI retinotopic 
mapping of four control subjects and one patient that had the portion of the right primary 
visual cortex representing the upper left visual field deafferented following a stroke. The deaf- 
ferented cortex was activated by stimuli presented to the lower left visual field. 


A reorganization of adult visual cortex was also reported By Dilks et al (2007) in 
a patient in which a stroke impaired the fibers that provide input to the region of the 
primary visual cortex that represents the upper left visual field (Dilks, Serences, 
Rosenau, Yantis, & McCloskey, 2007), as a result, the patient was blind in that part 
of the visual field. During a preliminary test, the authors noticed that the patient re- 
ported a distorted perception: stimuli presented to the lower left visual field ap- 
peared elongated towards the blind upper visual field, indicating that the deafferent- 
ed region of the primary visual cortex was responding to stimuli presented to the ad- 
jacent lower visual field. To test this hypothesis Dilks et al (2007) performed a stand- 
ard fMRI retinotopic mapping in primary visual cortex, and found that the large ar- 
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ea of V1 that normally responds to visual stimulation in the upper left visual field 
was activated by stimuli presented in the lower left visual field, clearly demonstrating 
a plastic rearrangement of primary visual cortex organization (Figure 3.1.4B). Taken 
together, these results demonstrate a plastic large-scale reorganization of adult visual 
cortex both in animals and humans following visual cortex deafferentation due to 
retinal or neural injury and challenge the view that considers neuroplasticity as a 
property limited to the sensitive period. 


3.1.2.2. Young and Adult visual cortical plasticity during recovery from blindness. 


In the previous paragraphs we have reviewed experimental evidence describing 
the existence of critical periods early in life, that is a precise temporal window during 
which the visual cortex of mammals is extremely plastic: even a short period of visu- 
al deprivation occurring during the critical period causes a permanent damage to the 
visual cortex organization. The critical period does not end abruptly, but it rather 
gradually closes, with longer periods of deprivation being necessary to cause visual 
cortical reorganization, for this reason, the critical period is now often defined as 
sensitive period (in the previous paragraphs, in fact, we have used the two terms in- 
differently, for review see Elman et al., 1996). As anticipated before, it is now clear 
that there is not one single sensitive period for vision: different visual capabilities 
show different sensitive periods, i.e. different duration for experience-dependent 
normal development and plasticity (Blakemore, 1988; Harwerth, Smith, Duncan, 
Crawford, & von Noorden, 1986; W. Singer, 1988). For example, in the monkey’s 
brain different sensitive periods have been found for scotopic and photopic sensitivi- 
ty (3 months and 6 months respectively), for spatial contrast sensitivity (18-24 
months) and binocularity (up to 2 years), Harwerth, Smith, Duncan, Crawford, & 
von Noorden, 1986. 

Not only different aspects of vision show different sensitive periods, but they al- 
so follow different developmental timecourses. At birth, infant humans possess only 
limited visual capabilities: they can only see low spatial frequencies (0.75 cpd) and 
some colours, they can discriminate between round and straight countours and be- 
tween orthogonal orientations, they can detect a face-like pattern and, finally, they 
can distinguish biological motion from scrambled motion (Adams, Maurer, & Davis, 
1986; Atkinson, Hood, Wattam-Bell, Anker, & Tricklebank, 1988; Fantz, 1963; 
Fantz, Ordy, & Udelf, 1962; Simion, Regolin, & Bulf, 2008; Slater, Morison, & Som- 
ers, 1988). During the first 6 months of life, vision improves rapidly, so that colour 
vision reaches adult-like levels and acuity improves fivefold (Franklin & Davies, 
2004; Maurer & Lewis, 2001a, , 2001b). In the next paragraphs we will detail the de- 
velopmental timecourses as well as the sensitive periods for different aspects of vi- 
sion, the two, in fact have been proposed to be closely related. The “Detroit model” 
proposed by Levi (1993), claims that there is a direct relationship between the devel- 
opmental pace of a visual capability and its susceptibility to the detrimental effects of 
visual deprivation, according to the principle borrowed from economics that “first 
hired, last fired”. According to the Detroit model the faster an aspect of vision devel- 
ops, the less it will suffer from abnormal visual experience (Levi & Carkeet, 1993). 

Sensitive periods have been mostly studied n animals, for depriving children of 
visual information is forbidden for obvious ethical reasons. A natural experiment 
allowing the investigation of sensitive periods in humans is provided by congenital 
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and developmental cataract, a treatable visual disease due to a clouding of the crys- 
talline lens. Testing children surgically treated (by replacing the opaque crystalline 
lens with a contact lens having an adequate refractive power) for unilateral or bilat- 
eral cataracts dense enough to block all patterned vision to reach the retina is a 
unique opportunity to study the effects of visual deprivation on the development of 
different aspects of vision. From studies involving children treated for congenital or 
developmental cataracts three different kinds of sensitive periods emerged: the sensi- 
tive period of visually driven normal development (the period during which visual in- 
put drives the normal development of the visual cortex), the sensitive period for 
damage (the period during which the visual system is vulnerable to the effects of vis- 
ual deprivation, this period can occur even after that normal development is 
achieved) and the sensitive period for recovery (the period when the visual cortex can 
recover from the effects of visual deprivation), Lewis & Maurer, 2005. 

Another interesting aspect that can be investigated by studying changes in visual 
capabilities in patients treated for cataracts is the role of competitive interactions be- 
tween the eyes in mediating different aspects of vision. Confronting the effect of bi- 
lateral versus unilateral cataracts, in fact, allows the investigation in humans of the 
effects of monocular and binocular deprivation. As anticipated in the previous para- 
graphs, animal studies have demonstrated that monocular deprivation provokes 
more severe damages on the primary visual cortex organization than binocular dep- 
rivation, with a severe rearrangement of cortical connectivity being observed after 
monocular deprivation in which the deprived eye loses the ability of driving cortical 
neuronal activity (Blakemore, 1988; Crawford, Pesch, von Noorden, Harwerth, & 
Smith, 1991; Harwerth, Smith, Paul, Crawford, & von Noorden, 1991; Hubel, 
Wiesel, & LeVay, 1977; LeVay, Wiesel, & Hubel, 1980; Wiesel & Hubel, 1965a). This 
has been interpreted as being the result of uneven competition between the weak 
signal of the deprived eye with the strong signal from the fellow non-deprived eye 
during and after deprivation, with probably Hebbian competition underlying the 
pruning of deprived eye synapses (Maurer & Lewis, 2001b). In animals, the conse- 
quences of this uneven competition between the signals of the two eyes can be re- 
duced by depriving the fellow eye during the sensitive period (Blakemore, Garey, & 
Vital-Durand, 1978; Wiesel & Hubel, 1965b), similarly, children treated for unilat- 
eral cataract undergo extensive patching of the “good” eye after the surgery and 
throughout the first 5-7 years of life. 

In the next paragraphs we will review experimental evidence on young and older 
children, as well as adults treated for cataracts, describing the effects of visual depri- 
vation on different visual capabilities and determining the different sensitive periods 
for those capabilities. We will also compare the effects of unilateral versus bilateral 
visual deprivation on different aspects of vision, examinig the importance of compe- 
tition between the eyes for those aspects. 


3.1.2.2.1. Effects of early visual deprivation on the development of basic visual 
capabilities: visual acuity and contrast sensitivity. 
Infants’ contrast sensitivity and acuity is very poor: newborns’ acuity is 40 times 


worse than adults’. After a rapid improvement during the first six months of life, 
visual acuity increases more slowly during the next 4 years of life (Figure 3.1.5A) and 
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does not reach adult levels until 6-7 years of age (Figure 3.1.5B), Ellemberg, Lewis, 
Liu, & Maurer, 1999; Mayer et al., 1995; Mayer & Dobson, 1982. A method to inves- 
tigate visual contrast sensitivity and acuity in babies is that of “preferential looking”, 
that takes advantage of the fact that infants prefer to look at something patterned ra- 
ther than at uniform images. The method consists in presenting contemporaneously 
in front of the baby two displays, one containing a grating of a certain contrast and 
spatial frequency, the other containing a uniform grey background of the same mean 
luminance. From a hole under the displays the experimenter (who is unaware of 
which of the two displays contains the patterned image) takes note of the time spent 
by the baby looking at one or the other image. The contrast and spatial frequency of 
the grating are varied until the infant shows no preference for one or the other dis- 
play, indicating that he cannot discriminate the grating from the uniform back- 
ground (Mayer et al., 1995; Mayer & Dobson, 1982). 

The scarce visual acuity at birth (newborns cannot discriminate spatial frequen- 
cies higher than 1 cpd), is thought to reflect the immaturity of the retinal cones, with 
the rapid improvement found during the first 6 months of life likely due to the de- 
velopment of cones in the fovea (Banks & Bennett, 1988; Candy & Banks, 1999). The 
subsequent slow development of normal contrast sensitivity during childhood is 
thought instead to reflect the refinement of retinal and cortical connectivity and 
general visual cortical architecture (Huttenlocher, 1984; Huttenlocher, de Courten, 
Garey, & Van der Loos, 1982; Kiorpes, Kiper, O'Keefe, Cavanaugh, & Movshon, 
1998). 
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Figure 3.1.5. The development of visual acuity and contrast sensitivity during infancy and 
childhood. Adapted from (Ellemberg, Lewis, Liu, & Maurer, 1999). 

(A) Development of visual grating acuity tested with the method of preferential looking from 
birth to 48 months of age, reviewed in (Maurer & Lewis, 2001a), from which the figure is 
adapted. (B) Changes in contrast sensitivity from the 4 to 7 years of age. The development of 
normal contrast sensitivity is achieved at 7 years of age. 


Maurer et al (1999) found that soon after the removal of congenital bilateral cat- 
aracts (10 minutes after the restoration of vision following surgery), patients’ visual 
acuity in both eyes was similar to that of newborns, independently from the patients’ 
age, that ranged from 1 to 9 months (Maurer, Lewis, Brent, & Levin, 1999), and was 
therefore worse than the aged-matched control babies. This results indicates that the 
rapid increase of visual acuity occurring during the first 6 months of life is driven by 
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the visual input. Interestingly, a rapid improvement was observed 1 hour following 
the end of visual deprivation, with patients’ visual acuity reaching the level of typical 
6-weeks old infants, confirming the importance of patterned visual input in driving 
the development of visual acuity. This results indicate that the visual system was not 
quiescent during visual deprivation, it is possible that, in infants with congenital cat- 
aracts, spontaneous retinal activity (for review see Katz & Shatz, 1996) is sufficient to 
preserve the cortical network implied in grating acuity. Furthermore, the rapid in- 
crease of visual acuity observed within 1 hour from restoration of vision indicates 
that the cortical neurons are ready to respond to the visually driven activity (Maurer, 
Lewis, Brent, & Levin, 1999). 

Despite the incredibly rapid initial improvement in visual acuity shown by in- 
fants treated for congenital cataracts, longitudinal studies (Ellemberg, Lewis, 
Maurer, Lui, & Brent, 1999; Lewis, Maurer, & Brent, 1995; Maurer, Ellemberg, & 
Lewis, 2006; Mioche & Perenin, 1986) on those patients revealed that the improve- 
ment involves solely low spatial frequencies (that recover contrast sensitivity to the 
level of normal control group), while sensitivity to mid and high spatial frequencies 
does not increase after the initial enhancement (Figure 3.1.6A). Contrast sensitivity 
to high spatial frequencies in 5 years old children treated for congenital cataracts is 
even not measurable (Ellemberg, Lewis, Maurer, & Brent, 2000). This result was also 
confirmed for letter acuity, that shows an asymptotic deficit in patients treated for 
congenital cataracts (Birch, Stager, Leffler, & Weakley, 1998). The only case in which 
a complete recovery of contrast sensitivity to higher spatial frequencies (and there- 
fore of visual acuity) is observed for children treated for congenital cataracts within 
the first 10 days of life (Kugelberg, 1992; Lundvall & Kugelberg, 2002). That sensitiv- 
ity to low spatial frequencies is spared after early visual deprivation is in line with the 
predictions of the Detroit model, for low spatial frequencies are the only spatial fre- 
quencies visible at birth, sensitivity to higher spatial frequencies developing later. 
The Detroit model also holds for other basic aspects of vision. For example, critical 
flicker fusion, that reaches adult-like levels as early as two months of age (Regal, 
1981) is preserved in children treated for congenital cataracts (Ellemberg, Lewis, 
Maurer, & Brent, 2000; Ellemberg, Lewis, Maurer, Lui, & Brent, 1999). 

That children treated for congenital cataracts early in life (within the first 9 
months of age), later show a deficit in high spatial frequency sensitivity is surprising, 
for the deprivation occurred well earlier than the time at which sensitivity to higher 
spatial frequencies matures. These phenomena are called “sleeper effects” (Maurer, 
Mondloch, & Lewis, 2007), and indicate that patterned vision during the first few 
months of life is crucial to set up the cortical neural architecture that will be finely 
tuned later in life. Moreover, these sleeper effects are likely to reflect deficits arising 
at the level of primary visual cortex: neurons in monkey’s primary visual cortex 
show sluggish responses and anomalous receptive fields (larger than normal) after 
binocular deprivation, cortical damage being even more severe if monkeys were mo- 
nocularly deprived. In contrast, the same deprived monkeys show normal retinal 
and thalamical responses (Blakemore & Vital-Durand, 1983; Crawford, Blake, Cool, 
& von Noorden, 1975; Crawford, Pesch, von Noorden, Harwerth, & Smith, 1991). 
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Figure 3.1.6. Grating acuity and sensitivity to global motion following treatment for unilateral 
and bilateral cataracts. Adapted from (Ellemberg, Lewis, Maurer, Brar, & Brent, 2002) 

(A) Grating acuity following the removal of unilateral or bilateral congenital and develop- 
mental cataracts. The dashed line represents the average acuity of a control group of normal 
observers. 

(B) Coherence thresholds for global motion discrimination following the removal of unilat- 
eral or bilateral congenital and developmental cataracts. The dashed line represents the aver- 
age acuity of a control group of normal observers. 


More insight about sensitive periods in humans comes from studies on devel- 
opmental cataracts, that is, children having an early history of normal vision that de- 
veloped dense cataract after the first 3 months of life (Lewis & Maurer, 2005; 
Maurer & Lewis, 2001a, , 2001b; Vaegan & Taylor, 1979). Interestingly, after treat- 
ment these patients show a better grating acuity compared with patients treated for 
congenital cataracts (Figure 3.1.6A), even though, only the few of them that had a 
very late onset of the disease (more than 11 years of age), showed normal visual acui- 
ty. This result indicates that visual deprivation occurring before 5 years of age pro- 
vokes permanent deficits to grating acuity. The sensitive period for damage is even 
longer for letter acuity: asymptotic letter acuity is impaired for visual deprivation 
starting as late as 10 years of age. Regarding the sensitive period for recovery, longi- 
tudinal studies on children treated for congenital cataracts (Ellemberg, Lewis, 
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Maurer, Lui, & Brent, 1999; Lewis, Ellemberg, Maurer, & Brent, 2000; Magnusson, 
Abrahamsson, & Sjostrand, 2002), demonstrated that, between 5 and 7 years of age, 
recovery for grating and letter acuity was still possible. 

What role does competition between the eyes play in the development of basic 
visual properties? As anticipated in the previous paragraph, children treated for 
dense unilateral cataract undergo aggressive patching of the fellow eye (50 to 90% of 
the waking time until 5-7 years of age) in order to compensate for the uneven com- 
petition between the two eyes provoked by monocular deprivation. As expected 
from the evidence obtained in animal studies, children treated for monocular cata- 
racts show worse deficits in spatial vision compared to children treated for bilateral 
cataracts (Figure 3.1.6A), deficits that are inversely correlated with the time of patch- 
ing of the fellow eye, though, even the most aggressive patching could not complete- 
ly restore vision in the deprived eye when unilateral visual deprivation occurred dur- 
ing the first two months of life (Jeffrey, Birch, Stager, Stager, & Weakley, 2001). The- 
se deficits include: grating acuity during infancy, asymptotic acuity and contrast sen- 
sitivity (Ellemberg, Lewis, Maurer, & Brent, 2000; Lewis, Maurer, & Brent, 1995; 
Lundvall & Kugelberg, 2002; Mayer, Moore, & Robb, 1989; Tytla, Maurer, Lewis, & 
Brent, 1988). Moreover, among the good patchers, visual acuity and contrast sensi- 
tivity was better in children treated earlier in life (Birch, Stager, Leffler, & Weakley, 
1998; Birch & Stager, 1996; Birch, Swanson, Stager, Woody, & Everett, 1993). 

Interestingly, the different severity of the visual deficits observed for unilateral 
deprivation compared to bilateral deprivation does not appear until the first year of 
age (Lewis, Maurer, & Brent, 1995; Mayer, Moore, & Robb, 1989). During the first 
month after restoration of vision, in fact, the recovery in visual acuity observed for 
patients treated for unilateral cataracts is comparable with that observed for patients 
treated for bilateral cataracts (Maurer, Lewis, Brent, & Levin, 1999), suggesting that 
the competitive interactions in primary visual cortex start to recover at around one 
year of age. Moreover, the sensitive period for damage observed in patients treated 
for unilateral cataracts is similar of that observed for binocular deprivation. Finally, 
deficits in spatial vision due uneven competition between the eyes caused by unilat- 
eral cataract have also been found in the fellow eye, that show a slightly worse visual 
acuity and contrast sensitivity than normal (Ellemberg, Lewis, Maurer, & Brent, 
2000; Lewis, Maurer, Tytla, Bowering, & Brent, 1992; McCulloch & Skarf, 1994; 
Thompson, Moller, Russell-Eggitt, & Kriss, 1996). Taken together, these results indi- 
cate that balanced activity of both eyes during the first months of life is necessary for 
the development of optimal visual acuity. 


3.1.2.2.2. Effects of early visual deprivation on the development of high level vis- 
ual capabilities: global motion and global form sensitivity. 


In the previous paragraph we have reviewed experimental evidence regarding 
the effects of early visual deprivation on basic visual features that are processed in 
primary visual cortex. How does early visual deprivation affect activity of higher lev- 
el visual areas? In this paragraph we will focus on experiments showing how sensi- 
tivity to global motion and global form changes in patients treated for bilateral or 
unilateral cataracts early in life. 

Sensitivity to the direction of global motion requires the integration of local 
motion signals (processed in primary visual cortex) over space and time, this pro- 
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cess requires the activity of extrastriate visual areas, including the motion-sensitive 
middle temporal (MT) extrastriate cortex (Maunsell & Newsome, 1987; Newsome 
& Pare, 1988), that is connected with V1 and projects to higher level areas of the 
superior temporal and ventral intraparietal cortex (Ungerleider & Desimone, 
1986). One efficient way to test sensitivity to the direction of global motion is us- 
ing random dots kinetograms (Newsome & Pare, 1988), in which the observer has 
to judge the global direction of motion in a display containing dots randomly posi- 
tioned with only a part of them moving in a coherent direction, the others moving 
in random directions (Figure 3.1.7A). To do the task the observer must integrate 
local motion signals over the visual field. The proportion of coherently moving 
dots is varied until a “coherence threshold” is found, that is the minimum propor- 
tion of coherently moving dots needed to discriminate the direction of global mo- 
tion. In normal children, coherent thresholds at 6 weeks of age are 36% (Banton & 
Bertenthal, 1996), sensitivity then improves over years, reaching adult-like values 
at around 6 years of age for dots moving at high speed (Ellemberg, Lewis, Maurer, 
Brar, & Brent, 2002), while for dots moving at lower speed mature sensitivity to 
the direction of global motion is not reached until the age of 10-11 years (Ellem- 
berg et al., 2004; Gunn et al., 2002). 

In children treated for bilateral congenital cataracts the perception of global mo- 
tion is highly compromised, these patients in fact show coherence thresholds 5 times 
higher than control normal subjects (Figure 3.1.7A, Figure3.1.6B), indicating that 
normal visual experience during the first 5 months of life (the average time of depri- 
vation in this group of patients) is necessary to develop sensitivity to the direction of 
global motion (Ellemberg, Lewis, Maurer, Brar, & Brent, 2002). This, again, is a 
sleeper effect, for normal development of global motion perception occurs well after 
the deprivation time sufficient to produce permanent alterations. That early normal 
experience is crucial for the development of sensitivity to global motion was con- 
firmed by testing patients who developed dense bilateral cataracts later in life, with a 
deprivation onset starting from 3 months of age. These patients in fact showed nor- 
mal coherence thresholds (Figure 3.1.7A), despite the abnormal visual acuity (Fig- 
ure3.1.6A), Ellemberg, Lewis, Maurer, Brar, & Brent, 2002. These results indicate 
that the sensitive period for damage for sensitivity to global motion is very short, 
with deprivation starting as soon as 3 months of age leaving coherence thresholds 
unaltered. 

Interestingly, the impairment in sensitivity to the direction of global motion was 
much smaller (3-fold) in patients treated for congenital unilateral cataracts, com- 
pared with patients treated for bilateral congenital cataracts (Figure 3.1.7A), and a 
similar result was obtained for sensitivity to global form (Figure 3.1.7B), a visual ca- 
pability that is thought to be processed in the extrastriate area V4 (H.R. Wilson, 
1999), even though in the case of global form, thresholds for monocular deprivation 
were only 1.6 times smaller than those for binocular deprivation (Lewis et al., 2002). 
These results indicate that some aspects of vision mediated by extrastriate areas in- 
volve collaborative rather than competitive interactions between the eyes following 
early visual deprivation. This could be due to the fact that after the first stages of vis- 
ual analysis (primary visual cortex), information about the eye of origin is not pre- 
served, and inputs from the two eyes converge, for in those higher level visual areas 
large portions of the visual field are represented, cortical neurons having large recep- 
tive fields. Moreover, the deficits found in patients treated for bilateral congenital 
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cataracts were less severe for global form compared to global motion perception, 
showing that, for high-level aspects of vision the Detroit model (Levi & Carkeet, 
1993) does not hold, for sensitivity to global form show a slower developmental 
timecourse than sensitivity to global motion (Ellemberg, Lewis, Maurer, Brar, & 
Brent, 2002; Lewis et al., 2004). Therefore, according to the predictions of the De- 
troit model, global form perception should have been more adversely affected by 
visual deprivation. 
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Figure 3.1.7. Sensitivity to global motion and global form in patients treated for congenital or 
developmental cataracts. 

(A) Coherence thresholds to global motion displays in patients treated for unilateral and bi- 
lateral cataracts, if deprivation started after the first 3 months of life (developmental cata- 
racts), patients showed normal coherence thresholds, while patients treated for binocular 
congenital cataracts showed abnormal global motion sensitivity. Unilateral congenital visual 
deprivation slightly affected sensitivity to the direction of global motion. (B) Thresholds for 
global form discrimination in patients treated for congenital bilateral and unilateral cataracts. 
After binocular deprivation sensitivity to global form was affected more severely than after 
monocular deprivation. Adapted from (Maurer, Lewis, & Mondloch, 2005). 


In general, the experimental evidence on children treated for cataracts re- 
viewed in the previous paragraphs clearly demonstrates that visual input during 
early infancy is fundamental to set up and preserve the optimal neural architec- 
ture underlying different aspects of vision. In the absence of visual input, the 
neurons dedicated to these visual capability may not develop properly, being 
pruned by Hebbian competition mechanisms eliminating the connections re- 
ceiving weak inputs. Another possibility (that is not mutual exclusive with the 
previous one), is that the neural network dedicated to the lost aspects of vision 
could be recruited for the processing of other sensory modalities during the dep- 
rivation period (as detailed in the previous chapter, the visual cortex of blind pa- 
tients is colonized by other sensory modalities, including audition and touch). 
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The developmental failure of different visual capabilities following early visual 
deprivation could be therefore due to the fact that the requisite neural architec- 
ture supporting them is not longer available. 


3.1.2.2.3. The Project Prakash: learning to see. 


Prakash in Sanskrit means “light”, that is what the Dr Shroff's Charitable Eye 
Hospital in New Dehli has been doing in India during the last decade in collabora- 
tion with the neuroscientist Pavan Sinha (Figure 3.1.8B): giving back light to blind 
children and young adults having treatable sight problems (Mandavilli, 2006). Ac- 
cording to Orbis International (a non-profit organization), one third of the total 
blind people in the world live in India and because of poverty, more than 60% of 
those blind children die before adulthood (Mandavilli, 2006). The Prakash project 
restores sight performing free surgery on the congenitally blind children, the majori- 
ty of which are born with corneal opacities or dense cataracts (Figure 3.1.8A). Most 
patients treated with the project Prakash are older than the standard patients treated 
for congenital cataracts and constitute a unique occasion to study the effects of pro- 
longed visual deprivation on the development of the visual system, as well as the sen- 
sitive periods for different aspects of vision in a time window rarely explored by oth- 
er studies. The first report on the Prakash project, in fact regarded a 29-years old 
man (subject S.K.) who regained vision thanks to the prescription of appropriate 
spectacles to correct his congenital aphakia (the absence of the crystalline lens, that 
is almost totally absorbed in the chambers). 

One of the first reports from the project Prakash described the recovery of visual 
abilities in a 34 years old woman with an early history of dense congenital bilateral 
cataracts (Ostrovsky, Andalman, & Sinha, 2006). S.R.D underwent surgery at the age 
of 12 years, and therefore had 22 years over which she had the possibility of acquir- 
ing visual function learning to see. As expected from animal studies and evidence on 
children treated for cataracts (reviewed in the previous paragraphs), S.R.D. had poor 
visual acuity (20/200), showing that visual restoration after the end of the sensitive 
period does not result in the recovery of basic aspects of vision. The team guided by 
Pavan Sinha performed several visual tests on S.R.D. involving basic form perception 
(shape matching, visual memory, matching transformed shapes, depth from config- 
ural cues and image segmentation) and face perception (face/non face discrimina- 
tion, face localization, face matching, gaze direction judgment and gender classifica- 
tion). Surprisingly, these authors found that, despite the prolonged period of visual 
deprivation starting from birth, S.R.D. performed as well as control normal observ- 
ers in all of the visual tasks (Ostrovsky, Andalman, & Sinha, 2006). The spared visual 
capabilities shown by S.R.D. were the result of an extensive learning and were not 
available soon after surgery. This result indicates that recovery of high level visual 
function is still possible via learning during adulthood, demonstrating that adult vis- 
ual cortex retains a high degree of neuroplasticity. 
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Figure 3.1.8. The Prakash Project. 

(A) Adapted from (Sinha & Held, 2012). Two examples of sight disease treated by the team: 
corneal opacities (left panel) and dense cataracts (right panel). 

(B) Adapted from (Mandavilli, 2006). Pavan Sinha testing the visual capabilities of one of the 
children treated in the hospital. 


In another report, the visual capabilities of 3 subjects from the project Prakash 
has been studied longitudinally from the restoration of sight up to 18 months later 
(Ostrovsky, Meyers, Ganesh, Mathur, & Sinha, 2009). One of the patients tested was 
S.K., whose case has been described above. The other two patients were two children 
who underwent surgery for dense bilateral congenital cataract, one (P.B.) at the age 
of 7 years, the other (J.A.) at the age of 13 years. In these patients, as in the case of 
S.R.D. visual acuity recovered only slightly and remained very poor. The three sub- 
jects were then tested soon after surgery for high level aspects of vision: static visual 
parsing (counting simple shapes presented on a display), object recognition (naming 
50 common objects) and dynamic visual parsing (counting simple shapes with addi- 
tional motion cues). All of the subjects showed good proficiency in enumerating ge- 
ometrical shapes when these were presented alone or with other non-overlapping 
shapes. When the different shapes presented in the same display overlapped, observ- 
ers were not able to do the task and perceived all closed loops and regions of uni- 
form luminance or color as separate objects (Figure 3.1.9A). This result indicates 
that patients whose vision was restored late in life (after the closure of the critical pe- 
riod), have a profound deficit in integrating contours into a global image, resulting 
in a overfragmentation of the images. Similarly, these patients also showed a poor 
performance in the complex objects naming task, where they perceived regions of 
different colors of luminance as distinct objects (Ostrovsky, Meyers, Ganesh, Ma- 
thur, & Sinha, 2009). 
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Figure 3.1.9. Visual parsing abilities of three patients from 3 up to 18 months after the resto- 
ration of vision. Adapted from (Ostrovsky, Meyers, Ganesh, Mathur, & Sinha, 2009). 

(A) Patients’ performance to static visual parsing tasks in which they had to count the num- 
ber of objects presented on the display. When geometrical shapes overlapped subjects were 
unable to perform the task because they overfragmented the visual scene. 

(B) Patients’ performance to dynamic visual parsing tasks. Motion cues allowed correct seg- 
regation of the overlapping visual objects. 

(C) Patients’ performance to static visual parsing tasks measured from 10 to 18 months fol- 
lowing visual restoration. Each of them improved in segmenting the objects, indicating that 
they learn this visual ability over time. 
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Interestingly, in the dynamic visual parsing task, where the instruction was the 
same as in the static visual parsing experiment (count the objects on the display), the 
patients responded correctly in the majority of the cases (Figure 3.1.9B), indicating 
that introducing motion cues enabled the correct segmentation of visual objects (Os- 
trovsky, Meyers, Ganesh, Mathur, & Sinha, 2009). Patients recently treated for visual 
disease used motion information to segregate objects from the background, while in 
static viewing conditions they showed a severe integrative impairment. This result is 
in line with a previous report by Fine et al (2003) that we will discuss in the next par- 
agraph (Fine et al., 2003). The authors also speculate that the earlier development of 
motion sensitivity observed in normal children is the factor that allows the subse- 
quent correct development of static segmentation (Ostrovsky, Meyers, Ganesh, Ma- 
thur, & Sinha, 2009). 

Finally, when the same three patients were tested on static visual parsing later af- 
ter restoration of vision (from 10 to 18 months), an important improvement of per- 
formance was observed for all of them (Figure 3.1.9C). This result indicates that high 
level visual function as object parsing was acquired by the patients later after the 
treatment, the delay being necessary for the subjects to “learn how to see” (Ostrov- 
sky, Meyers, Ganesh, Mathur, & Sinha, 2009). As in the case of S.R.D., high level vis- 
ual capabilities were acquired in adult subjects via learning and needed a prolonged 
time to recover, probably because of the more limited plasticity of adult visual cortex 
compared to the juvenile cortex. In any case, the evidence described above clearly 
questions the idea of a strict restriction of plasticity within the sensitive period, 
demonstrating that high level visual capabilities can be recovered via learning also 
during adulthood. 


3.1.2.2.4. The man who learnt to see. 


Almost a decade ago, Fine et al (2003) reported the outstanding case of a 43 
years old man (M.M.), who reacquired vision in one eye after 40 years of deprivation 
(Fine et al., 2003). When he was 3 and a half years old, in fact, M.M. completely lost 
one eye and had a severe thermal damage to the cornea of the other eye, such that he 
only had some light perception in that eye, but no patterned experience. Almost 40 
years later, M.M. was operated (stem-cell transplant) and recovered some vision in 
the right eye. Following surgery, in line with the results observed with the other pa- 
tients whose vision was reacquired after prolonged deprivation, M.M. showed poor 
visual acuity, with spared contrast sensitivity only for low spatial frequencies (maxi- 
mum visible spatial frequency at high contrast 1 cpd). 

Thorough examination revealed that M.M.’s right eye retina was not damaged, 
suggesting a central origin of his visual impairment for mid and high spatial fre- 
quencies. This was confirmed by fMRI results: already at low spatial frequencies, 
BOLD responses in V1 were one fifth compared to V1 responses of control normal 
observers, when visual stimuli spatial frequency exceeded 1 cpd, MM’s BOLD re- 
sponses fell off dramatically and were near to zero. M.M.’s visual acuity was retested 
longitudinally for 2 years following surgery and did not improve within that time, in 
line with the evidence that early visual deprivation permanently impairs contrast 
sensitivity to mid and high spatial frequencies discussed in the previous paragraphs. 
The case of M.M. is however peculiar, in fact, at the age of 3 and a half years (when 
he became blind), his visual acuity should have been around 25 cpd, way more than 
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the sensitivity recovered after treatment. This result indicates that prolonged depri- 
vation had degraded M.M.’s visual cortex resolution. 

When tested for higher-level visual functions (Figure 3.1.10), M.M. showed op- 
timal performance in simple form tasks, as well as good orientation sensitivity and 
color perception. He had no problems in segmenting texture patterns based on lu- 
minance contrast, but showed a slight deficit in integrating texture elements into a 
global shape. He also could not resolve illusory contours, such as the Kanisza trian- 
gle, unless the form of the triangle was outlined. Similarly, he could not interpret 
prospective cues and was unable to resolve the Necker cube. In line with the results 
obtained with the patients of the Prakash project (Ostrovsky, Andalman, & Sinha, 
2006; Ostrovsky, Meyers, Ganesh, Mathur, & Sinha, 2009), M.M. showed overfrag- 
mentation mistakes: he interpreted two geometrical shapes overlapping in transpar- 
ency as three distinct objects (even though he could easily resolve occlusions), and 
he showed a extensive deficit in identifying common objects and unfamiliar faces 
(Fine et al., 2003). 
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Figure 3.1.10. Psychophysical visual tasks and performance 5 months after treatment. 
Adapted from (Fine et al., 2003). 

Three different aspects of vision were investigated administering psychophysical visual tests 
to the patient M.M. and a control group of normal subjects: form recognition, depth percep- 
tion and motion sensitivity. 


Interestingly, despite the important deficits shown in object recognition, M.M.’s 
performance was very good in most of the motion tasks: he could detect well the di- 
rection of simple and complex plaid motion and could segment texture patterns 
based on motion, as well as identifying dynamic Glass patterns from random noise. 
Not only he showed spared motion sensitivity per se, but he could also proficiently 
use motion cues to resolve transparency and three-dimensional shapes, such as the 
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Necker cube, that he could not resolve in static conditions. The pattern of fMRI acti- 
vation to high-level visual stimuli reflected M.M.’s psychophysical performance: 
BOLD responses in the motion-sensitive area V5/MT were comparable to that of 
normal controls, while the visual areas usually devoted to the processing of faces (fu- 
siform gyrus) and complex objects (lingual gyrus) were not activated by their usually 
preferred visual stimuli (Fine et al., 2003). 

Taken together, the case of M.M. suggests that not only early visual deprivation 
(soon after birth) is important to develop normal visual function, but also visual ex- 
perience beyond 3 years of age is necessary to achieve optimal visual processing. 
Prolonged visual deprivation starting at 3 and a half years of age caused long-lasting, 
if not permanent damages to visual cortical organization and resolution, with the ex- 
ception of motion processing. As detailed in the previous paragraphs, motion sensi- 
tivity develops earlier than form processing, and, as predicted by the Detroit model 
(Levi & Carkeet, 1993), is more robust to the effect of visual deprivation, for at the 
age of 3 and a half years, the neural network dedicated to motion processing could 
be more established and hard-wired. Another possible explanation suggested by Fine 
et al (2003) for the pattern of visual deficits shown by M.M. is that the mechanisms 
underlying the analysis of complex objects could remain plastic for a longer period 
to allow learning of novel shapes throughout life. 


3.1.2.3. Adult human cortical plasticity due to transient visual deprivation or 
prolonged adaptation. 


In the previous paragraphs we have described how young and adult visual cortex 
can plastically adapt to abnormal sensory input following sensory loss the duration 
of which spanned from a few months to decades. Another way of approaching the 
study of visual cortical plasticity is that of investigating whether the adult visual sys- 
tem can adapt to transient modifications of the visual input, either through tempo- 
rary deprivation of visual input, or via prolonged adaptation. In this paragraph we 
will review some recent experiments that addressed this issue and demonstrated re- 
sidual plasticity of adult human visual cortex in response to transient modifications 
of the visual environment, such as light deprivation, contrast reduction and pro- 
longed orientation-specific deprivation. 


3.1.2.3.1. Short-term light deprivation induces plastic changes of adult human 
visual cortical excitability. 


In 2000, Boroojerdi et al investigated whether a transitory period of light depri- 
vation was able to modulate adult visual cortical excitability (Boroojerdi et al., 2000). 
Their hypotheses were that a brief period (180 minutes) of light deprivation would 
result in enhanced excitability and in an augmented activation in response to visual 
stimulation of the occipital cortex of adult human observers. To test these hypothe- 
ses, the authors directly stimulated with transcranial magnetic stimulation (TMS) 
and measured variation in the BOLD signal in the occipital cortex of adult partici- 
pants undergoing a period of light deprivation (Boroojerdi et al., 2000). 

When applied on the occipital cortex, TMS can induce the perception of flashes 
of light in the absence of visual stimulation, these TMS-evoked visual sensations are 
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called “phosphenes” (Marg & Rudiak, 1994). By varying the intensity of the tran- 
scranial magnetic stimulation, one can determine the so called “phosphene thresh- 
old”, that is the minimum TMS intensity necessary to elicit phosphenes. Variations 
in phosphene thresholds are thought to convey information about the excitability of 
the occipital (visual) cortex (Afra, Mascia, Gerard, Maertens de Noordhout, & 
Schoenen, 1998; Aurora, Ahmad, Welch, Bhardhwaj, & Ramadan, 1998). To moni- 
tor possible changes in visual cortical excitability provoked by light deprivation, Bo- 
roojerdi et al (2000), measured phosphene thresholds in adult human participants at 
regular intervals of 45 minutes (Figure 3.1.11A.), before, during and after a period of 
light deprivation (180 minutes). Importantly, in the absence of visual deprivation, 
phosphene thresholds tested in the same intervals did not significantly varied (Bo- 
roojerdi et al., 2000). Interestingly, a significant reduction of phosphene thresholds 
was observed during the deprivation period, the largest reduction occurring within 
the first 90 minutes (Figure 3.1.11A). To test the timecourse of phosphene thresh- 
olds recovery, Boroojerdi et al (2000) also had observers undergoing light- 
deprivation for 90 minutes (that is the time necessary to achieve the maximum var- 
iation in phosphene thresholds), and found that thresholds reverted to normal val- 
ues at around 120 minutes following re-exposure to normal vision (Boroojerdi et al., 
2000). 
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Figure 3.1.11. Changes in adult human occipital cortical excitability and activation during 
transient periods of light-deprivation. Adapted from (Boroojerdi et al., 2000). 

(A) Diagram of the paradigm and results of the TMS experiment. Phosphene thresholds were 
measured during and after a period (180 or 90 minutes) of light deprivation in normal adult 
observers. During the first 90 minutes of light deprivation, phosphene thresholds significantly 
decreased and did not return to baseline levels until 120 following re-exposure to normal vi- 
sion. 

(B) Results of the fMRI experiment. BOLD activation was measured in the occipital cortex 
during and after 60 minutes of light deprivation. Both the number of voxels and the intensity 
of the signal in response to photopic stimulation increased after light deprivation and were 
still higher than baseline measurements 30 minutes after the offset of visual deprivation. 


Light deprivation not only induced changes in cortical excitability that resulted 
in decreased phosphene thresholds, but also increased neural activity during photic 
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stimulation, as measured by fMRI BOLD signal: after 60 minutes of light depriva- 
tion, an increase in both the number of voxels activated and in the intensity of the 
BOLD signal at the level of the occipital cortex (but not in LGN) was observed (Bo- 
roojerdi et al., 2000). This increased neural activation did not return to baseline 30 
minutes after re-exposure to light (Figure 3.1.11B). These results indicate that the 
adult visual cortex still retains a form of experience-dependent plasticity, for a brief 
period of light deprivation is able to increase occipital cortical excitability (reflected 
in the decreased phosphene thresholds after deprivation) and neural activation (re- 
flected in the augmented hemodynamic response of the occipital cortex). 

In a subsequent study by the same group (Boroojerdi, Battaglia, Muellbacher, & 
Cohen, 2001), the mechanisms underlying this form of experience-driven plasticity 
were investigated by repeating the TMS experiment in adult human observers after 
the administration of different drugs that are known to interfere with synaptic plas- 
ticity: lorazepam (a benzodiazepine that increases GABAergic inhibition by acting as 
positive allosteric of GABAA receptors), dextrometorphan (a substance that blocks 
NMDA receptors involved in mechanisms of long term potentiation, LTP), scopola- 
mine (an antagonist of acetylcholine muscarinic receptors) and lamotrigine (an an- 
tiepileptic that does not influence LTP, but blocks voltage-gated Na* and Ca?* chan- 
nels). Interestingly, the administration of all of these drugs, but lamotrigine, pre- 
vented the light-deprivation-induced increase of cortical excitability observed in 
drug-naives observers and measured via phosphene threshold changes (Boroojerdi 
et al., 2000). From this result, the authors concluded that the transient neuroplastici- 
ty produced by light deprivation in adult human visual cortex is regulated at least by 
three different mechanisms: balance between intracortical GABAergic inhibition 
and cholinergic excitation, activation of NMDA receptor and mechanisms of long- 
term potentiation (Boroojerdi, Battaglia, Muellbacher, & Cohen, 2001). Interesting- 
ly, as detailed in the previous paragraphs, the same neural and molecular mecha- 
nisms have been found to be important for critical period regulation in animals. 


3.1.2.3.2. Plastic alterations of the adult human visual cortex activity during pro- 
longed contrast reduction. 


As described in the previous paragraphs, patients with dense cataracts experi- 
ence prolonged contrast deprivation, with only some spared light perception, but no 
pattern information reaching the retina. An interesting study by Kwon et al (2009) 
investigated the effects of prolonged (4 hours) exposure to contrast reduction on 
visual contrast sensitivity and early neural activation (BOLD signal in V1 and V2) in 
adult human observers (Kwon, Legge, Fang, Cheong, & He, 2009). These authors 
hypothesized that extended contrast reduction would cause compensatory modifica- 
tions in the adult visual system, resulting in an increased response gain in visual cor- 
tex that would be reflected behaviourally in an improve of contrast discrimination 
thresholds. 

To explore this hypothesis, Kwon et al (2009) had a group of adult observers 
wearing contrast-reducing goggles for 240 minutes (main experiment) and a control 
group of adult observers wearing neutral-density goggles for the same period of time 
(control experiment). Before and after the period of contrast reduction, contrast dis- 
crimination thresholds were tested in the two groups of observers for seven different 
contrast pedestals (0%, 0.3%, 1%, 1.6%, 3.3%, 8.3% and 16.6% of filtered contrast), in 
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order to obtain thresholds-vs-contrast (TvC) functions (Legge & Foley, 1980; 
Nachmias & Sansbury, 1974). Moreover, {MRI BOLD activation in V1 and V2 was 
measured for the two groups of participants for different contrasts (1%, 3.3%, 8.3% 
and 16.6% of filtered contrast), in order to obtain neuronal contrast response func- 
tions (CRF) from the BOLD activation in early visual areas (Boynton, Demb, Glover, 
& Heeger, 1999; Zenger-Landolt & Heeger, 2003). Figure 3.1.12A shows a diagram 
of the experimental paradigm. 
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Figure 3.1.12. The effect of prolonged contrast reduction on contrast discrimination thresh- 
olds and neuronal contrast response functions in V1 and V2 of adult human observers. 
Adapted from (Kwon, Legge, Fang, Cheong, & He, 2009). 
(A) Adult observers underwent 4 hours of exposure to low contrast in one eye, while a trans- 
lucent eyepatch was worn on the other eye, before and after contrast reduction contrast dis- 
crimination thresholds and BOLD signal in V1 and V2 were measured (main experiment). A 
control group of subjects wore neutral filters instead of contrast reducing ones and performed 
the same tasks. 


(B) After 4 hours of contrast reduction, contrast discrimination thresholds improved, leading 
to a downward shift of the TvC function and activity in V1 and V2 was enhanced for the 
whole range of contrasts tested (1-16.6% of filtered contrast). 


The contrast-reducing filter attenuated contrast by a factor of 3 (0.5 log unit) 
and minimized blur, resulting in a small acuity reduction (less than 0.2 logMAR) 
and in a luminance reduction of a factor of 2 (0.3 log unit). The filters were created 
with diamond powder and casting acrylic (Pelli, 1987). The contrast-reducing gog- 
gles used in the main experiment had this contrast-reducing filter mounted over the 
observer’s dominant eye and a translucent occluder that did not allowed patterned 
vision and was matched with the contrast-reducing filter for luminance attenuation 
mounted over the fellow eye. The neutral density filter goggles used in the control 
experiment were similar to the contrast-reducing goggles, but had a neutral-density 
filter matched in luminance attenuation with the other filters mounted over the 
dominant eye (Kwon, Legge, Fang, Cheong, & He, 2009). In both experiments, psy- 
chophysical and fMRI measurements were obtained with observers wearing the con- 
trast-reducing goggles. 

Figure 3.1.12B shows the psychophysical TvC functions and the neuronal CRF 
functions measured before and after four hours of exposure to low-contrast for the 
main experiment (upper panels) and for the control group wearing the neutral den- 
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sity goggles for the same amount of time (lower panels). Prolonged adaptation to 
low contrast resulted in lowered contrast discrimination thresholds and in increased 
fMRI BOLD activation (neuronal CRFs) compared with measurements obtained be- 
fore contrast-reduction. Importantly, no difference between pre- and post- 
deprivation measurements was observed in the control experiment, indicating that 
the behavioural and neural changes observed were a consequence of the exposure to 
low contrast and did not depend on fatigue. 

Interestingly, the pattern of results obtained was consistent with prolonged con- 
trast reduction resulting in augmented response gain in V1 and V2. Neuronal CRFs 
were in fact steeper after long-term adaptation (with an increase of the maximum 
neural response of a factor of 1.3) and the TvC were shifted downwards, indicating 
improved contrast discrimination thresholds for all pedestal contrast tested (Figure 
3.1.12B). Finally, the cortical origin of the observed effect was confirmed by the fact 
that the adaptive changes produced by prolonged adaptation to low contrast showed 
full interocular transfer (Kwon, Legge, Fang, Cheong, & He, 2009). 

Taken together, these results indicate that the adult visual system reacts to the 
reduction of incoming information due to prolonged exposure to low contrast by 
increasing cortical response gain in order to maximize neural response to weaker 
visual input. These compensatory changes in adult visual cortex activity increase the 
evidence in favour of a residual plastic potential of adult visual cortex, more than 
previously thought. 


3.1.2.3.3. Orientation-specific visual deprivation induces changes in visual sensi- 
tivity. 


During development the visual cortex of mammals is highly vulnerable to the ef- 
fects of visual deprivation. Not only ocular dominance plasticity is observed after pe- 
riods of monocular deprivation, but also deprivation to a specific orientation can al- 
ter the organization of the developing visual cortex. If, during the sensitive period, 
kittens are reared in the dark with daily exposure only to a certain orientation (e.g. 
vertical or horizontal), the number of neurons in primary visual cortex responding 
to the deprived orientation is dramatically reduced and kittens permanently lose the 
ability of seeing the deprived orientation (Blakemore & Cooper, 1970; Hirsch & Spi- 
nelli, 1970; Sengpiel, Stawinski, & Bonhoeffer, 1999; Tanaka, Ribot, Imamura, & 
Tani, 2006). Using an innovative experimental setup, Zhang et al (2009) investigat- 
ed the effects of selective deprivation to a specific orientation on visual perception of 
adult humans (Zhang, Bao, Kwon, He, & Engel, 2009). In this study, observers wore 
a head-mounted video camera feeding into a laptop, that processed online the imag- 
es from the camera, filtering them in real time and driving a head-mounted display 
in which the filtered images of the external world were presente. In this way, observ- 
ers were free to navigate and interact with the external environment. The filtering 
resulted in deprivation of a specific orientation, e.g. the vertical orientation as shown 
in Figure 3.1.13. in which a picture of the setup and the resulting natural and filtered 
images are reported. 
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Figure 3.1.13. Orientation-specific deprivation enhances visual sensitivity to the deprived ori- 
entation. Adapted from (Zhang, Bao, Kwon, He, & Engel, 2009). 

Adult participants experienced 4 hours of selective deprivation to a specific orientation, using 
a head-mounted display filtering online the incoming images from a head-mounted video 
camera. After 4 hours of selective deprivation, observers’ discrimination thresholds for the 
deprived orientation significantly decreased during the following 40 minutes, indicating im- 
proved sensitivity. 1 hour of deprivation was ineffective in producing the performance gain 
observed for 4 hours of deprivation. 


Interestingly, Zhang et al (2009) found that, following 4 hours of selective depri- 
vation to a specific orientation, sensitivity to the deprived orientation significantly 
improved, with lower discrimination thresholds for the deprived orientation being 
obtained during the first 40 minutes following deprivation offset. After shorter-term 
(one hour) deprivation instead discrimination thresholds did not differ from base- 
line measurements (Figure 3.1.13.). Moreover, the improvement in sensitivity ob- 
served after 4 hours of selective deprivation went back to baseline measurements 
when discrimination thresholds were measured 24 hours afterwards (Zhang, Bao, 
Kwon, He, & Engel, 2009). These results indicate that visual deprivation to a specific 
orientation results in a compensatory increase in responsiveness of neurons repre- 
senting the deprived orientation in early adult visual cortex. 

A following study by the same group further investigated the effects of orienta- 
tion-specific deprivation (Bao & Engel, 2012). In this study, the effect of orientation 
deprivation of different durations (1, 4 and 8 hours) on visual perception was meas- 
ured by quantifying the strength and duration of the tilt aftereffect (TAE). Following 
deprivation (obtained with the experimental setup described above and shown in 
Figure 3.1.13) observers were asked to adjust the orientation of two overlapping 
gratings (plaid pattern) oriented at +45°, until they appeared to be squared. 

Figure 3.1.14A shows that the strength of the tilt aftereffect increased with in- 
creasing duration of deprivation, with size of the tilt aftereffect increasing in a nearly 
linear fashion with the logarithm of the adaptation duration (Bao & Engel, 2012). 
Not only the strength, but also the duration of the TAE increased for longer periods 
of deprivation. For the 4 and 8 hours of deprivation conditions, in fact the curves, 
obtained by measuring repeatedly over time the size of the TAE, asymptoted to a 
positive effect (Figure 3.1.14A). The timecourse of the TAE after deprivation offset 
was well fit both by power and exponential decay functions, for it showed a peak 
soon after the end of deprivation, with a rapid decay and a constant value thereafter 
(Figure 3.1.14A). 
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Figure 3.1.14. Deprivation and adaptation to a specific orientation show similar timecourses. 
Adapted from (Bao & Engel, 2012). 

(A) Observers were adapted to real-world images deprived of the vertical orientation for differ- 
ent amounts of time (1, 4 or 8 hours). After orientation deprivation, a strong tilt aftereffect was 
measured: the strength and duration of the effect depended on the duration of the deprivation. 
(B) Observers were adapted for 4 hours to real-world images in which the contrast of the vertical ori- 
entation was enhanced. After adaptation the measured tilt aftereffect was in the opposite direction 
compared with that observed for deprivation, but showed a similar dynamics, with the effect rapidly 
decaying over the first minute of visual stimulation and then asymptoting to a positive effect. 


The authors hypothesized that the same compensatory neural mechanisms, but 
acting in opposite directions underpinned the effect of deprivation and adaptation. In 
the case of deprivation, the visual system would react by increasing neural responsive- 
ness (gain), in order to make sure that none of the impoverished information get lost. 
In the case of adaptation, instead, the same mechanisms would decrease neural gain in 
order to avoid response saturation to highly contrasted visual stimulation. In both cas- 
es, this mechanism would allow the accurate signaling of small changes in environ- 
mental contrast. To test this hypothesis, Bao & Engel (2012) tested the tilt aftereffect 
after 4 hours of prolonged adaptation to enhanced contrast of a specific orientation 
with the same altered reality setup. They found that prolonged adaptation to high con- 
trast vertical orientations resulted in a robust TAE (in the opposite direction compared 
to the deprivation-induced TAE) that decayed to a positive asymptote with a dynamics 
extremely similar to that observed for orientation-specific deprivation (Figure 
3.1.14B), confirming the hypothesis that same neural mechanisms underlie the two 
phenomena (Bao & Engel, 2012). Finally, the effect of 4 hours of deprivation was also 
found to vanish after 15 minutes of exposure to normal visual scenes (deadaptation). 
Interestingly, the effect of deprivation, annulled by the deadaptation, was fully restored 
by exposing observers to only 8 minutes of orientation-selective deprivation. The au- 
thors interpreted these results as an evidence in favour of two distinct mechanisms op- 
erating for short-term and long-term visual adaptation (Bao & Engel, 2012). 

Taken together, the results described in this paragraph demonstrate that the adult 
human visual cortex retains a high degree of experience-driven plasticity that operate 
also in response of selective deprivation of a specific orientation. Moreover, these find- 
ings reinforce the evidence in favour of a compensatory reaction of the adult visual 
system to the lack of information, that results in an increased responsiveness of the 
neurons whose input had been impoverished in order to compensate for the signal at- 
tenuation and optimally convey the visual information surviving deprivation. 
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In the next paragraphs we will present two experimental studies in which we used 
binocular rivalry as a tool to investigate residual neuroplasticity of the adult human 
visual system after short-term visual deprivation in one eye. 

In the first study we found that, after 150 minutes of monocular deprivation, visual 
perception during binocular rivalry was strongly biased in favour of the deprived eye, 
with an effect on the onset of rivalry being measurable for up to 90 minutes following 
re-exposure to normal binocular vision. Moreover, by testing apparent contrast before 
and after the same patching procedure, we also showed that following monocular dep- 
rivation stimuli presented to the previously deprived eye appeared to be stronger than 
stimuli presented to the non-deprived eye by a factor of 1.36. 

In the second study we expanded the results obtained in the first report by show- 
ing that the effect of monocular deprivation on the dynamics of binocular rivalry is 
stronger and more long-lasting when chromatic rather than luminance-modulated 
gratings are tested, with an effect on sustained rivalry being measurable for up to 180 
minutes following eyepatch removal. 

Finally, we will also present some preliminary data from an experiment in which 
we measured binocular rivalry in amblyopic children during five months of occlu- 
sion therapy. These preliminary report indicates that binocular rivalry can be effi- 
ciently used as a tool to monitor neuroplasticity during the recovery of visual func- 
tion in the amblyopic eye driven by the occlusion of the fellow eye. 


3.2. Brief periods of monocular deprivation disrupt ocular balance in human 
adult visual cortex. 


Neuroplasticity is a fundamental property of the developing mammalian visual 
system, with residual potential in adult human cortex (Karni & Bertini, 1997). A short 
period of abnormal visual experience (such as occlusion of one eye) before closure of 
the critical period has dramatic and permanent neural consequences, reshaping visual 
cortical organization in favour of the non-deprived eye (Berardi, Pizzorusso, & Maffei, 
2000; Wiesel & Hubel, 1963b). We used binocular rivalry (Tong, Meng, & Blake, 2006) 
- a sensitive probe of neural competition - to demonstrate that adult human visual 
cortex retains a surprisingly high degree of neural plasticity, with important perceptual 
consequences. 150 minutes of monocular deprivation strongly affected the dynamics 
of binocular rivalry, unexpectedly causing the deprived eye to prevail in conscious per- 
ception twice as much as the non-deprived eye, with significant effects for up to 90 
minutes. Apparent contrast of stimuli presented to the deprived eye was also in- 
creased, suggesting that the deprivation acts by up-regulation of cortical gain-control 
mechanisms of the deprived eye. The results suggest that adult visual cortex retains a 
good deal of plasticity that could be important in reaction to sensory loss. 


3.2.1. Results and Discussion 
We investigated the effects of monocular deprivation on the dynamics of binocu- 
lar rivalry in adult humans. Seven observers wore a translucent eye-patch on one eye 


for 150 minutes, then viewed a dichoptic binocular-rivalry display with horizontal 
grating patches presented to one eye and vertical to the other (Figure 3.2.1a), report- 
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ing by continuous key-press which pattern they perceived. Figure 3.2.1b shows the 
effect of monocular deprivation on the phase durations during rivalry, separately for 
the deprived (black symbols) and non-deprived (orange symbols) eyes. The bars 
show the mean phase durations (normalized to each subject’s baseline condition, 
then averaged over subjects) for five contiguous 3-minute sessions (see Figure 3.2.2. 
for raw data). Immediately after eyepatch removal, phase durations of the deprived- 
eye pattern increased by 53%, while those of the non-deprived eye decreased by 24%, 
a two-fold difference between eyes (the effect larger when patching the preferred 
than the non-preferred eye: factors of 2.6 cf 1.7). The difference in phase duration 
between the two eyes decayed steadily over time, but remained significant 15 min 
after eyepatch removal (paired t-test, n=11, a=0.025, p<0.01). Despite the strong bi- 
as towards the deprived eye, observers reported that the quality of binocular rivalry 
did not change after deprivation, with continued alternations between the two mo- 
nocular images, with almost no periods of fused images. 
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Figure 3.2.1. Effect of monocular deprivation on mean phase durations. 

(A) After 150 minutes of patching, five consecutive sessions of Binocular Rivalry were record- 
ed, where observers viewed orthogonally oriented Gabor Patches, vertical to one eye and hori- 
zontal to the other. (B) Relative phase durations (expressed as a fraction of the mean baseline 
phase duration for each observer) of the two visual stimuli, as a function of time elapsed from 
the removal of the eyepatch, for the stimulus presented to the deprived (black) and non-de- 
prived (light grey) eyes. The points show individual data, the bars group averages: bar symbols 
at right show average s.e.m.) (C) The average proportion of times observers reported seeing the 
stimulus presented to the deprived eye, expressed as a function of time elapsed from the onset 
of each experimental session, smoothed within a Gaussian window of time constant 1 second. 
For at least 6 minutes after deprivation, the probability stays above chance, indicating that it was 
always more probable to see the stimulus presented to the deprived eye. 
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Figure 3.2.1C shows the average instantaneous probability of seeing the de- 
prived-eye stimulus, as a function of elapsed time, averaged over all subjects and ses- 
sions and smoothed with a Gaussian window of time constant 1 s. Monocular depri- 
vation biased the trace consistently towards the deprived eye. The initial percept of 
each testing session (a sensitive parameter of rivalry (Carter & Cavanagh, 2007)) was 
most strongly biased: in the first session the deprived eye prevailed in 93% of first- 
phases, remaining at 78% after 15 minutes. Even after 90 minutes, the bias towards 
the deprived eye remained significant (64%: sign test, n = 49, a = 0.025, p<0.001). 
After deprivation, gratings viewed by the deprived eye appeared of higher contrast 
than those by the non-deprived eye, by a factor of 1.36 (on average), but detection 
thresholds were virtually unaffected. The increase in apparent contrast is qualitative- 
ly consistent with the relatively shorter binocular phase periods to the non-deprived 
eye (Levelt’s second law), but the amount of increase is quantitatively insufficient to 
explain the imbalance in rivalry (see Figure 3.2.3.). 


Mean Phase Duration 


Deprived Eye (sec) 


Non-Deprived Eye (sec) 


Figure 3.2.2. Raw Dominance Phase Durations after Monocular Deprivation. 

For each experimental session, mean dominance phase duration of gratings presented to the 
deprived eye are plotted against mean phase duration of gratings presented to the nondeprived 
eye, for various durations after removal of the eyepatch. Different grey levels distinguish the dif- 
ferent observers, filled and open symbols refer respectively to preferred and nonpreferred eye 
patching. For the first 15 minutes after the removal of the eyepatch, all symbols lie upon the bi- 
sector of the graph, indicating that mean phase duration of the stimulus presented to the de- 
prived eye was longer than that of stimuli presented to the non deprived eye. 90 minutes after 
re-exposure to binocular vision, balance between the eyes was restored, with all symbols clus- 
tered around the bisector, with a little bias for observers’ eye preference. It is well known that 
mean phase durations during binocular rivalry show a large inter-individual variability(Carter 
& Pettigrew, 2003): here the range is from 0.7 to 7 seconds. 


112 


Claudia Lunghi 


Within a specific critical period, mammalian visual cortex is highly vulnera- 
ble to the effects of visual experience: but it is often assumed that mammalian 
adult visual systems, including humans, show little plasticity after the closure of 
this period (Berardi, Pizzorusso, & Maffei, 2000). However, there is a growing 
bulk of literature suggesting that the adult human visual system retains some 
plasticity (Boroojerdi, Battaglia, Muellbacher, & Cohen, 2001; Karni & Bertini, 
1997; Klink, Brascamp, Blake, & van Wezel, 2010; Zhang, Bao, Kwon, He, & En- 
gel, 2009), and some of these alterations, such as perceptual learning, may have 
long lasting effects. A recent study (Klink, Brascamp, Blake, & van Wezel, 2010) 
has shown that prolonged periods of binocular rivalry (> 35 min) increases the 
frequency of mixed or fused binocular rivalry, suggesting that exposure to in- 
congruent signals between the two eyes may decrease the reciprocal interocular 
inhibition that causes rivalry, promoting fusion rather than competition between 
the two incompatible images. Our current study complements and expands on 
that of Klink et al.: 150 minutes of monocular deprivation causes a two-fold 
prevalence of the deprived eye, with measurable effects lasting up to 90 minutes, 
revealing a further form of deprivation-induced plasticity in adult human cortex, 
which boosts signal strength of the deprived eye. 

That deprivation causes an increase in apparent contrast suggests that the 
deprivation-induced changes to the dynamics of binocular rivalry may be me- 
diated by up-regulating contrast gain-control mechanisms in the deprived eye 
in response to the period of reduced signal-strength. The increase in gain- 
control probably occurs at the cortical level, given that the patch was translu- 
cent (causing no dark-adaption), and the long timecourse of the effects (while 
retinal and LGN adaptation timecourses are short, Baccus & Meister, 2002). 
However, the changes were not by themselves sufficient to explain the preva- 
lence of the deprived eye, implicating additional mechanisms specific of binoc- 
ular rivalry. Gain-control mechanisms are ubiquitous in mammalian brains, 
and in several neurological dysfunctions, subtle changes of contrast gain have 
been observed (Dakin, Carlin, & Hemsley, 2005), indicating that the excitato- 
ry/inhibitory balance that modulates gain control mechanisms could be partic- 
ularly susceptible to abnormal developmental events. Increasing cortical con- 
trast gain of the deprived eye may reflect neuroplastic mechanisms attempting 
to optimize weak or absent information. 

Overall, the present results demonstrate that abnormal visual experience 
can drastically affect adult vision. The effects probably reflect transient chang- 
es, not long-lasting as for perceptual learning, in neuronal circuitry in primary 
visual cortex, possibly related to calibration of the system. That binocular rival- 
ry can reveal significant neural plasticity in adult visual cortex suggests that it 
could become an important non-invasive tool to assess reorganization of the 
visual cortex in visual pathologies like amblyopia and to monitor excita- 
tion/inhibition balance during plastic changes in human. 
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Figure 3.2.3. Effect of monocular deprivation on apparent contrast. 

Apparent contrast of stimuli presented after MD for two observers. (A) Observers matched the 
contrast of a test visual stimulus presented to one eye to that of a “standard” presented to the 
other eye, in the same spatial location at all contrast tested (0.2, 0.4, 0.6 and 0.8). Before depriva- 
tion (upper graph), all points lie on the bisector, showing the precision and accuracy of observer 
matches. After deprivation (lower graph), stimuli to the deprived-eye (black symbols) need a 
lower contrast to match the contrast of the non-deprived eye stimuli (light grey symbols), indi- 
cating that they appeared of higher contrast. The slope of the linear fit of the data were 0.82 and 
0.89 indicating a slight non-linear compression. The black and orange arrows plot the contrast 
discrimination thresholds, virtually unaffected by. (B) mean phase durations of 3 observers av- 
eraged for the first 9 minutes after removal of the eyepatch for stimuli of balanced contrast of 
0.75 and for stimuli with unbalanced contrast by a factor of 1.36 (0.9 and 0.66). The phase dura- 
tion of the low-contrast stimuli is increased by 10%, but clearly insufficient to explain the factor- 
of-two effect of monocular deprivation. 
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3.3. Long-term effects of monocular deprivation revealed with binocular rivalry 
gratings modulated in luminance and in color 


The developing sensory brain is highly plastic (Pascual-Leone, Amedi, Fregni, & 
Merabet, 2005), allowing it to self-calibrate and to adapt to the environment. Plastic- 
ity in humans, and indeed all mammals, is regulated within a clearly defined critical 
period (Wiesel & Hubel, 1963b): early visual deprivation, such as early untreated 
congenital cataracts, provokes permanent deficits both for basic visual functions 
such as visual acuity and high-level functions such as shape and depth perception 
(Fine, Smallman, Doyle, & MacLeod, 2002; Fine et al., 2003; Levi, McKee, & 
Movshon, 2011; Maurer, Lewis, & Mondloch, 2005; Ostrovsky, Andalman, & Sinha, 
2006). Plasticity in young infants is so profound that in the congenitally blind, other 
sensory modalities invade the visual cortex, which starts to respond to tactile (Sadato 
et al., 1996) and auditory (Roder, Stock, Bien, Neville, & Rosler, 2002) stimulation. 
Competition between the monocular inputs is a crucial factor contributing to the 
plasticity of the developing visual system: binocular deprivation affects visual cortex 
organization of animals less than monocular deprivation (Wiesel & Hubel, 1965a) 
and humans with unilateral cataracts show more severe deficits than those with bi- 
lateral cataracts (Lewis, Maurer, & Brent, 1995). 

It has been generally assumed that after closure of the critical period, the brain 
becomes relatively hard-wired, with little or no experience-dependent plasticity (Be- 
rardi, Pizzorusso, & Maffei, 2000; Fine et al., 2003; Hensch, 2004; Maurer, Lewis, & 
Mondloch, 2005). Recent evidence, however, has questioned this assumption, and 
the degree of neuroplasticity in adult mammals is now a debated issue. In adult ani- 
mals, ocular dominance plasticity can be restored by increasing excitation or by de- 
creasing inhibition in the CNS (Harauzov et al., 2010; Maya Vetencourt et al., 2008), 
confirming the importance of the excitation-inhibition balance in determining visu- 
al cortical plasticity. Although in adult humans there is no evidence of ocular domi- 
nance plasticity, adult visual cortex shows a residual plastic potential, as demonstrat- 
ed for fine properties of vision like perceptual learning (Karni & Bertini, 1997), ori- 
entation tuning (Bao & Engel, 2012; Zhang, Bao, Kwon, He, & Engel, 2009), contrast 
discrimination (Kwon, Legge, Fang, Cheong, & He, 2009), multisensory processing 
(Merabet et al., 2008) and binocular fusion (Klink, Brascamp, Blake, & van Wezel, 
2010). 

We recently introduced a novel technique to study plasticity in adult humans 
(Lunghi, Burr, & Morrone, 2011): we combined binocular rivalry with monocular dep- 
rivation - the classic paradigm used to investigate ocular dominance plasticity - to 
demonstrate that adult human visual cortex retains a surprisingly high degree of neu- 
ral plasticity. When two incompatible images are displayed separately to the eyes, they 
do not merge into a single percept, but compete for visual awareness, resulting in ine- 
luctable perceptual alternations with only one image dominating perception at a time, 
only to be supplanted by the previously suppressed one. This form of bistable percep- 
tion, called binocular rivalry (Blake & Logothetis, 2002), probably reflects reciprocal 
inhibition of the two rival images (Tong, Meng, & Blake, 2006), making it an optimal 
tool to study visual competition in early visual processing (Haynes & Rees, 2005). In 
our previous study we demonstrated that a short period of monocular deprivation 
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(150 minutes) had important consequences on the dynamics of binocular rivalry be- 
tween luminance-modulated gratings: following monocular deprivation the deprived 
eye strongly dominated visual perception over the non-deprived eye, with an effect be- 
ing measurable for up to 90 minutes following re-exposure to binocular vision. 

Here we extend this technique to study the effects of deprivation on binocular ri- 
valry on equiluminant chromatic stimuli, modulated in color (to favor the parvocellu- 
lar system) and compare these effects to those with luminance-modulated grating (re- 
analyzed from data of Lunghi, Burr, & Morrone, 2011). We show that monocular dep- 
rivation affects the dynamics of binocular rivalry for both luminance- and chromatic- 
modulated stimuli, but more so for chromatic stimuli, where it biases rivalry in favor 
of the deprived eye for at least 3 hours after 2.5 hours monocular deprivation. Brief pe- 
riods of deprivation (30 minutes) have very little effect on rivalry. 


3.3.1. Materials and Methods 


Observers 

Five observers (two males, mean age 24+0.8 years), including author CL, partici- 
pated in experiment with chromatic gratings, and seven observers (one male, mean 
age 26.7+2 years, all different except CL) participated in the experiment with achro- 
matic gratings. Four participants (one male, mean age 24.5+0.7 years, two who did 
not participate in other studies) participated in the short-term deprivation experi- 
ment. All had normal or corrected to normal vision, normal stereo acuity (Frisby 
stereotest Sasieni, 1978), normal color vision, and no strong eye preference. Partici- 
pants gave informed consent and were reimbursed for their time at the rate of 7€ per 
hour. The experiments were carried out along the principles laid down in the decla- 
ration of Helsinki, and the paradigm approved by the ethics committee of the Scien- 
tific Institute Stella Maris, and observers gave written informed consent. 


Apparatus and Stimuli 

The experiment took place in a dark and quiet room. Visual stimuli were gener- 
ated by the ViSaGe (CRS, Cambridge Research Systems) housed in a PC (Dell) con- 
trolled by Matlab programs. Equiluminant chromatic stimuli were displayed on a 
linearized monitor (Barco CDCT 6551) driven at a resolution of 987x777 pixels, with 
a refresh rate of 120 Hz. Achromatic stimuli were displayed on a 20-inch Clinton 
Monoray (Richardson Electronics Ltd., LaFox, IL) monochrome monitor, driven at a 
resolution of 1024x600 pixels, with a refresh rate of 120 Hz. Observers viewed the 
display at a distance of 57 cm through CRS Ferro-Magnetic shutter goggles that oc- 
cluded alternately the two eyes each frame. Responses were recorded through the 
computer keyboard. The eyepatch was made of a translucent plastic material that 
allowed light to reach the retina (attenuation 15%), but no pattern information, as 
assessed by the Fourier transform of a natural world image seen through the eye- 
patch. During the patching period observers were free to perform their normal activ- 
ities, such as working, reading, walking outside and having lunch. 

Chromatic stimuli were equiluminant sinusoidal gratings, made by summing 
magenta and cyan sinusoidal gratings of equal but opposite contrast, oriented 
obliquely at +45° (size: 2°, spatial frequency: 1.5). They were displayed on a uniform 
gray background (luminance: 32 cd/m’, C.I.E: 0.341 0.368) in central vision with a 
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central black fixation point; a common squared dark grey frame (size 2.5°) to facili- 
tate dichoptic fusion. Given that the blue gun was kept constant at 1,the ratio of the 
red luminance to the sum of the red and green luminance, R/(R+G), was used to de- 
termine the subjective equiluminant point of the subjects, evaluated by standard 
minimum flicker photometry. Points of equiluminance varied between 0.48 and 0.5 
for the five observers. To avoid local chromatic adaptation, we randomly shifted the 
phase of the visual gratings in one or the other direction at a rate of 0.3-0.5 Hz. The 
background was set at the mean value of the individual guns of the equiluminant 
grating, equiluminance between the gratings and the background was measured with 
the photometer (Konika Minolta). 

Achromatic stimuli were two Gaussian-vignetted sinusoidal gratings (Gabor 
Patches), oriented either vertically or horizontally (size: 20 = 1.5°, spatial frequency: 
3cpd, contrast: 75%), presented on a uniform background (luminance: 37.4 cd/m/?, 
C.LE: 0.442 0.537) in central vision with a central black fixation point and a com- 
mon squared frame to facilitate dichoptic fusion. 

For the equiluminant stimuli, luminance and C.I.E coordinates were 32 
cd/m’, C.I.E: 0.363 0.272 for the magenta grating and 32 cd/m’, C.I.E: 0.297 0.581 for 
the cyan grating. Cone contrasts along the axes were LM axis: L=8.5%, M=13.5%; S ax- 
is: S=77% (Smith & Pokorny, 1975). The chromaticities of the visual stimuli in a cone 
excitation space (MacLeod & Boynton, 1979) were: L/(L+M) = 0.61 and 
S/(L+M)=0.002 for the red grating and L /(L+M) = 0.7 and S/(L+M)=0.01 for the 
green grating. Presentations were alternated at the frequency of the shutter goggles, so 
each eye was presented with only one of the two stimuli. Monocular deprivation was 
achieved by having observers wear the translucent eye-patch for 150 minutes for the 
two main experiments and for 30 minutes in the short-term deprivation experiment. 

To test the effect of monocular deprivation on the achromatic and chromatic 
visual pathway, we used visual stimuli that elicited maximum responses of the two 
systems: high contrast achromatic Gabor patches with a spatial frequency of 3 cpd 
and equiluminant magenta/cyan oriented gratings with spatial frequency of 1.5 cpd, 
because responses to equiluminant sinusoidal gratings show low-pass characteristics 
(Kaplan, Shapley, & Purpura, 1988). A diagram of the equiluminant visual stimuli is 
reported in Figure 3.3.1A. The baseline mean phase duration of the two types of vis- 
ual stimuli was comparable both for the group average (Figure 1B). 


Task and Procedure 

In the experiment with luminance and chromatic gratings, each observer was 
measured separately eight times, patching each eye four times in pseudo random or- 
der. Each individual patching session was separated at least by 24 hours. We also 
measured baseline conditions for each observer before patching, yielding eight sepa- 
rate measurements. After patch removal, we measured binocular rivalry continuous- 
ly for 15 minutes, giving a short break every three minutes. For luminance gratings 
we measured a three-minute block of rivalry again at 90 minutes from patch remov- 
al, for chromatic gratings we measured three-minute blocks at 30, 45, 60 90 120, 150 
and 180 minutes. For short deprivation the procedure was the same, but we meas- 
ured only the first 15 minutes. 

Eye dominance was assessed operationally from binocular rivalry baseline re- 
cordings, with the dominant eye being the one that prevailed. Immediately after the 
removal of the eye-patch observers sat in front of the monitor wearing the shuttering 
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goggles and the first experimental session began. After a countdown, the binocular 
rivalry stimuli appeared. Subjects reported their perception (clockwise or counter- 
clockwise for the equiluminant gratings and horizontal or vertical for the achromatic 
gratings) by continuously pressing with the right hand one of two keys (left or right 
arrows) of the computer keyboard. As assessed in pilot studies and in debriefing ses- 
sions, mixed percepts were very rare, occurred for only very brief periods between 
perceptual transitions, and their frequency remained constant across conditions 
even after deprivation. Neither subject nor experimenter knew which stimulus was 
associated with which eye until the end of the session, when it was verified visually. 


3.3.2. Results 
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Figure 3.3.1. Mean Phase Durations of the different visual stimuli before and after monocular 
deprivation. (A) Othogonally oriented equiluminant gratings (S.F. 1.5 cpd, orientation +45°) 
modulated only in chromaticity along the L/M axis and achromatic Gabor patches (S.F. 3 cpd, 
orientation 0-90°) modulated only in luminance contrast (75%) were presented separately to the 
eyes through FE-Shuttering goggles. (B) Group mean phase durations did not differ for the two 
visual stimuli tested. (C-D) The average ratio between mean phase duration of stimuli presented 
to the deprived non-deprived eye of four measuraments of a single observer (preferred 
eyepatched) is plotted as a function of time elapsed from the removal of the eyepatch. (C) When 
luminance modulated gratings with different contrast (25% - 50% - 75%) were tested, after the 
first 15 minutes following eye-patch removal, the ratio between deprived eye and non-deprived 
eye mean phase duration did not statistically differ from baseline measurements. (D) When 
equiluminant stimuli were tested, the ratio between deprived and non-deprived eye duration 
significantly differred from the baseline for the whole period tested after monocular deprivation 
(180 minutes). Error bars represente +1 s.e.m. 


118 


Claudia Lunghi 


Two groups of subjects wore translucent patches for 150 minutes. After re- 
moval of eye-patch, binocular rivalry was tested at regular intervals with lumi- 
nance- or chromatic-modulated gratings. Data from observers tested with lumi- 
nance-modulated gratings have been reported briefly (Lunghi, Burr, & Morrone, 
2011), and were re-analyzed in this paper to allow a direct comparison with data 
from the new group of observers tested with chromatic gratings. Fig. 3.3.1 shows 
the results for one exemplary subject who performed both experiments (author 
CL). For both luminance- (Fig. 3.3.1C) and color- (Fig. 3.3.1D) modulation, the 
brief deprivation strongly affected dominance, biasing perception in favor of the 
deprived eye. The effects were stronger and more long-lasting for the chromatic 
than for the luminance gratings: on patch removal, binocular rivalry for chromatic 
gratings was three times more prevalent in the deprived than fellow eye, and the 
effect lasted for at least 180 minutes. Luminance gratings also biased rivalry to- 
wards the deprived eye, initially by a factor of two, lasting for about 30 minutes af- 
ter patch removal. 

As detailed in the methods section, the equiluminant gratings had lower cone 
contrast than the luminance gratings, 25% compared with 75%. To assess the im- 
portance of contrast, we re-measured the effect with luminance-modulated grat- 
ings of 50% and 25% contrast. As the results of fig. 3.3.1C show, contrast had very 
little effect on the bias in rivalry, either in the amplitude or the longevity, effective- 
ly ruling out reduced contrast as an explanation for the longevity of the effects 
with color gratings. 

Figure 3.3.2 shows the average results for all subjects (7 for the luminance 
condition, 5 for color). These average results are similar to those of the example 
subject of Fig. 3.3.1. For luminance gratings (black symbols), mean phase duration 
of the deprived eye increased by 56% on eye-patch removal, while that of the non- 
deprived eye decreased by 28%, a 2.3-fold difference between the eyes. Chromatic 
gratings (grey symbols) were similar, a 56% increase in the deprived eye, 27% de- 
crease in the non-deprived eye, yielding a factor of 2.3. The baseline measurements 
do not differ from 1 (implying perfect balance between the eyes). Following 150 
minutes of monocular deprivation, the ratio between the deprived and non- 
deprived eye mean phase duration was significantly (luminance gratings: paired t- 
test, N=7, t(6)= 6.28, p<0.001; chromatic gratings: paired t-test, N=5, t(4)= 4.19, 
p<0.05) biased in favor of the deprived eye (was therefore >1). The effect of mo- 
nocular deprivation was comparable for the two types of visual stimuli tested dur- 
ing the first 3 minutes following eye-patch removal, but followed different dynam- 
ics for luminance and chromatic visual stimuli. When luminance modulated grat- 
ings were tested, the effect of monocular deprivation on mean phase durations was 
only significant for data recorded during the first 15 minutes following re- 
exposure to binocular vision, data recorded 90 minutes after eye-patch removal 
clearly show that balance between the eyes was restored (paired t-test, N=7, t(6)= 
0.35, p=0.73). For chromatic gratings, rivalry was significantly biased in favour of 
the deprived eye for at least 3 hours following re-exposure to binocular vision 
(paired t-test, N=5, t(4)= 2.81 p<0.05). 180 minutes after removal of the eye-patch, 
mean phase duration of the deprived eye was 38% longer than that of the non- 
deprived eye. In addition, the difference between phase durations for luminance 
and chromatic-modulated stimuli recorded 12 minutes following re-exposure to 
binocular vision was statistically significant (t-test, t(10)=2.29, p<0.05), a differ- 
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ence that was confirmed also for data recorded 90 minutes after eye-patch removal 
(t-test, t(10)=2.93, p=0.015). The data are well fitted by a power function of the 
form: 


y = 1+ (a/log (t + 1) )^b 
(eq. 1) 

Where y is the magnitude of the effect, t time expressed in log and a and b free 
constants determining respectively amplitude and decay time. The half-life of the ef- 
fect, defined as the time at which the fitting curve reaches one half the initial effect 
(value 1.6, indicated by the dashed line in Figure 3.3.2) was 3.7 minutes for luminance- 
modulated gratings and 27.2 minutes for gratings modulated in chromaticity, showing 
that the decay of the effect was slower for chromatic stimuli by a factor of 7.3. 
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Figure 3.3.2. Effect of Monocular Deprivation on Binocular Rivalry Mean Phase Durations. 
The ratio between mean duration of the deprived and non-deprived eye is plotted as a func- 
tion of time elapsed from the removal of the eye-patch for luminance (gray symbols, average 
of 52 measures, 4 repetitions x 7 observers, only preferred eye was patched for one observer. 
(Data taken from Lunghi, Burr, & Morrone, 2011) and chromatic (black symbols, average of 
40 measures, 4 repetitions x 2 eyes x 5 observers) gratings. Error bars represent +1 s.e.m. The 
dashed line represent balance between the two eyes. Following 150 minutes of monocular 
deprivation phase duration is strongly unbalanced in favour of the deprived eye. The data are 
well fitted by a function of the form given in eq. 1 
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To investigate whether shorter periods of monocular deprivation have similar 
effects on binocular rivalry, we tested a third group of observers with only 30 
minutes deprivation, with luminance-modulated gratings. This short amount of 
deprivation had little effect on rivalry (Figure 3.3.3): during the first 3 minutes, per- 
ception was significantly biased towards the deprived eye (paired t-test, N = 4, 
t(3)=4.72, p<0.05), but the effect was much less than after 150 minutes of patching, a 
factor of only 1.26 compared with 2.3. Furthermore, the effect was significant only 
during the first 3 minutes after patch removal: data recorded later did not differ 
from baseline measurements. 
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Figure 3.3.3. Effect of Short-Term Monocular Deprivation on Binocular Rivalry Mean Phase 
Durations. 

The ratio between mean phase duration of the deprived and non-deprived eye is plotted as a 
function of time elapsed from the removal of the eye-patch for luminance-modulated gratings 
(average of 28 measures, 4 repetitions x 4 observers, only preferred eye was patched for one ob- 
server). The dashed line represents equal balance between the two eyes. Following 30 minutes of 
monocular deprivation phase duration is slightly unbalanced in favour of the deprived eye only 
during the first 3 minutes following eyepatch removal. Error bars represent +1 s.e.m. 


Rivalry is traditionally characterized by phase-duration distributions, which 
have a characteristic asymmetrical distribution, usually well approximated by a two- 
parameter (r, A) gamma distribution of the form: 


g(x) = (A*r x*(r — 1))/T (r) e(-4x) 

(eq. 2) 
where T is the gamma function, r is the shape parameter and À is the scale pa- 
rameter (Levelt, 1967). Figure 3.3.4 reports phase-duration distributions of the de- 
prived (black) and non-deprived (orange) eye (normalized for each observer to the 
baseline mean phase duration of that eye), and the relative gamma-distribution fits, 
for several 3-minute experimental blocks for luminance (Figure 3.3.4A) and chro- 

matic (Figure 3.3.4B) visual stimuli. 
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Figure 3.3.4. Phase-duration distributions of the deprived (black) and non-deprived (light grey) 
eyes, plotted separately for different 3-minutes experimental blocks for luminance (A, data tak- 
en from Lunghi, Burr, & Morrone, 2011) and chromatic (B) gratings after monocular depriva- 
tion. Phase-durations were normalized to the mean baseline phase-duration for each subject, 
because of the great inter-individual variability in mean phase duration (from 1 to 9 seconds for 
luminance gratings, from 2 to 6 seconds for chromatic gratings). Phase durations distributions 
are well fitted by a two-parameter (À, r) gamma distribution of the form given in eq.2. 
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The baseline distributions (top panels) are very similar for the two eyes, with 
similar values of r and À. After monocular deprivation, phase-duration distributions 
of the deprived eye became broader and shifted towards the right (indicating that, on 
average, all phase durations were longer), while the opposite held for phase duration 
distributions of the non-deprived eye, indicating that all phase durations were short- 
er. Nonetheless, the distributions maintained the typical gamma-like characteristics 
and were well fit by the gamma distribution. The separation between the phase dura- 
tion distribution of the deprived and that of the non-deprived eye was greater for 
chromatic than for luminance stimuli: 90 minutes after eye-patch removal, phase- 
duration distributions of the two eyes were identical for luminance-modulated grat- 
ings, while they clearly remained different for chromatic gratings. 

Figure 3.3.5 plots the ratio between r and À separately for the deprived eye (filled 
symbols) and non-deprived eye (open symbols), as a function of time from eye-patch 
removal. In line with the literature on the dynamics of binocular rivalry (De Marco, 
Penengo, & Trabucco, 1977), in the baseline measurements, \ and r of the same eye 
distribution were virtually identical, approximating unity. Monocular deprivation af- 
fected \ and r differently, particularly for chromatic gratings (black symbols): the 
shape parameter r remained basically unaltered, while the scale parameter À decreased 
for the deprived eye and increased for the non-deprived eye. This effect was just as 
prevalent 120 minutes after removal of eye-patch. For luminance-modulated Gratings 
(grey symbols), the differential effect on the two parameters was mostly apparent for 
the deprived eye, while for the non-deprived eye a slight difference between the two 
was noticeable only during the first 6 minutes of binocular vision. 
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Figure 3.3.5. Phase Duration Distribution Parameters. The ratio between the shape (r) and 
scale (A) parameter of the gamma distribution fits (eq. 2), plotted as a function of time 
elapsed from removal of the eyepatch, for luminance (grey symbols, data taken from Lunghi, 
Burr, & Morrone, 2011) and chromatic gratings (black symbols) and the deprived (filled 
symbols) and non-deprived (open symbols) eye respectively. Error bars represent +1 s.e.m. 


The analysis of mean phase durations and phase-duration distributions are 
standard in binocular rivalry. A more dynamic way of approaching the analysis of 
bistable perception is to track the probability of perceiving one or other stimulus 
over time (Lunghi, Burr, & Morrone, 2011; Mamassian & Goutcher, 2005). The ad- 
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vantage of this method is that it describes the dynamics of rivalry, providing a time- 
course of visual perception, while the analysis of phase durations does not take into 
account the order of the events during a period of observation (because of the as- 
sumption that phase durations are independently and stochastically distributed). 
This is important because it has been demonstrated that at least two different pro- 
cesses with different characteristics operate during binocular rivalry, one at the onset 
of rivalry and one during sustained observation (Carter & Cavanagh, 2007). 

We therefore computed the probability of perceiving the visual stimulus pre- 
sented to the deprived eye (averaged over 6s bins), as a function of time elapsed from 
rivalry onset, for each 3-minute experimental block. Figure 6 reports the timecourse 
of the probability of seeing the stimulus presented to the deprived eye for luminance 
modulated (Figure 3.3.6A) and for chromatic gratings (Figure 3.3.6B). The baseline 
probabilities oscillate constantly around chance level, indicating that the stimuli pre- 
sented to each eye were equally likely to be perceived. Monocular deprivation affect- 
ed both the onset of rivalry and the sustained level of rivalry, but in different ways 
for luminance and chromatic stimuli. The probabilities recorded after deprivation 
are well fit by an exponential decay function of the form: 


y = Ae(t/t) + y_0 
(eq. 3) 

Where y is the magnitude of the effect, A the maximum amplitude, t the decay 
constant and yo a lower asymptote. After an initial exponential decay, the probability 
asymptotes to a level (yo) above chance for all the testing sessions following depriva- 
tion when chromatic gratings are tested (Figure 3.3.6B), while for luminance modu- 
lated gratings the probability decays to chance level 90 minutes after eye-patch re- 
moval and only the bias on onset rivalry is present. When we directly compared the 
probabilities recorded 90 minutes after re-exposure to binocular rivalry for chro- 
matic and luminance gratings we found that for chromatic gratings the probability 
of seeing the stimulus presented to the deprived eye was systematically higher than 
for luminance gratings: taken together both the probabilities recorded during the 
first 25 seconds of viewing (onset effect) and those recorded during the following 
155 seconds (sustained effect) were significantly higher for chromatic gratings (boot- 
strap sign-test, 1,000,000 repetitions, Ho: chromatic > luminance, p<0.001 for the 
onset effect and p<0.05 for the sustained effect). 

For luminance gratings, monocular deprivation affected rivalry more at onset than 
during the sustained period: the asymptotic difference between deprived and non- 
deprived eye phase durations (i.e. the offset of the decay, yo) decayed rapidly for lumi- 
nance gratings to become insignificant 15 minutes after eye-patch removal, while for 
chromatic gratings the effect remained significant for the whole 3-hour period tested 
(Figure 3.3.7A). Conversely, the bias in onset rivalry (Figure 3.3.7B), followed a simi- 
lar timecourse for luminance and for chromatic gratings, even though the onset bias 
measured 90 minutes following patch removal was higher for chromatic than for lu- 
minance gratings (t-test, t(90) = -2.616, p <0.01). Moreover, for chromatic gratings, the 
onset bias was significantly higher than chance level after 180 minutes following patch 
removal (t-test, t(39)= 2.4655, p < 0.02). Monocular deprivation influences the dynam- 
ics of binocular rivalry in different ways, depending on the type of visual stimuli tested, 
being mostly effective on onset rivalry for achromatic stimuli and effective both on on- 
set and sustained rivalry for chromatic stimuli. This reinforces the suggestion that two 
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processes are at work with binocular rivalry (Carter & Cavanagh, 2007), and these are 
differently affected in luminance and chromatic gratings after monocular deprivation. 


A Luminance Gratings B Chromatic Gratings 


1.0 
0.51 sere. te e. A -2 0.92.2 FOR ROIO 
È iy ge st oO, tus eee 6 
Baseline Baseline 
"raaa TITLE re ee ee 
1.0 
T 
(00) 
2 
[us 
(È 
o 
A 05 
DI 
c ; ; 
D 1:3 min 1:3 min 
D Soe SS eee 
Mm 1:0 
— 
e) 
Pai 
— 
5 o oo °° e 
g A = 2 49%, -* ae e e 00, ® o° 
a 54 ---79-08-g----2--.. eo e eee 
à 
12:15 min 12:15 min 
erer e 
1.0 
A 0° ° = 
0.5 °__00° ae FR Pi et ES Rs Matt 
į EF 
87:90 min 87:90 min 


ma maaa an ea 
O 30 60 90 120 150 180 O 30 60 90 120 150 180 


Time from Rivalry Onset (sec) 


Figure 3.3.6. Average proportion of reported deprived eye dominance. 

The probability of perceiving the stimulus presented to the deprived eye expressed as a func- 
tion of time elapsed from the onset of different 3-min experimental blocks for luminance (A, 
data taken from Lunghi, Burr, & Morrone, 2011)and chromatic (B) gratings. The probability 
traces were computed by calculating the frequency of deprived eye dominance (sampling rate 
= monitor refresh rate, 120 Hz) in 6s bins, for every experimental session recorded (52 for 
luminance gratings, 40 for chromatic gratings). . The average probabilities across sessions are 
well fit by an exponential decay function given in eq. 3. Error bars represent 1+s.e.m for every 
6 sec bin. The average parameters of the fitting functions are reported in Figure 3.3.7. 
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Figure 3.3.7. Probability Trace Asymptote and Onset bias. 

A. The asymptote of the effect of deprivation, corresponding to the fitting parameter yo of eq. 
3, as a function of time elapsed from eye-patch removal. Error bars represent +1 s.e.m. B. The 
probability of seeing the stimulus presented to the deprived eye at the onset of rivalry. In both 
cases gray symbols refer to luminance and black symbols to chromatic gratings. The dashed 
lines in both graph represent chance level, that is no effect. Error bars represent +1 s.e.m. (for 
luminance gratings, data taken from Lunghi, Burr, & Morrone, 2011) 
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3.3.3. Discussion 


Within a specific critical period (Hubel & Wiesel, 1970; Wiesel & Hubel, 
1963b), the mammalian visual cortex is highly vulnerable to the effects of visual 
experience, but it is generally assumed that mammalian adult visual systems, in- 
cluding humans, show little plasticity after closure of this period (Berardi, Pizzo- 
russo, & Maffei, 2000; Fine et al., 2003; Hensch, 2004; Maurer, Lewis, & Mondloch, 
2005). Our results provide a clear demonstration that the adult human visual sys- 
tem retains a high degree of plasticity, far more than previously thought: two and a 
half hours of monocular deprivation impacts dramatically on the dynamics of bin- 
ocular rivalry, causing a two-fold dominance of the deprived eye, with measurable 
effects lasting up to 180 minutes, depending on the type of visual stimulation. Alt- 
hough the effect could in principle have a sub-cortical origin, we believe this un- 
likely, given that the patch was translucent (therefore causing no dark-adaption), 
and that retinal and geniculate alteration of neuronal discharge show a fast adapta- 
tion timecourse (Baccus & Meister, 2002; Solomon, Peirce, Dhruv, & Lennie, 
2004). 

The data reported here point to a plasticity of ocular dominance in adult hu- 
man visual cortex. Monocular deprivation is an effective technique to reveal plas- 
ticity, as it drives competitive Hebbian-like mechanisms, like those responsible for 
the major neural reorganization within the critical period (Mitchell & Sengpiel, 
2009). That following monocular deprivation the deprived eye is reinforced and 
wins the competition for visual awareness dominating rivalrous perception over 
the non-deprived eye is an unexpected result, for long-term monocular depriva- 
tion usually results in depression of deprived eye input (Wiesel & Hubel, 1963b). 
Boosting the signal of the deprived eye could be the first response of the visual sys- 
tem to the lack of information provoked by monocular deprivation, an attempt to 
optimize response to weak stimulation probably by homeostatically modulating 
contrast-gain mechanisms. Homeostatic bidirectional plasticity has been indeed 
observed in the mouse visual cortex, where increased responses of both deprived 
and non-deprived eye have been found after monocular deprivation (Mrsic-Flogel 
et al., 2007). The importance of competitive mechanisms for visual cortical plastici- 
ty as been confirmed by recent evidence (Xu, He, & Ooi, 2010) showing that per- 
ceptual learning (where the weak eye is reinforced and contemporarily the strong 
eye is suppressed) is able to reduce sensory eye dominance and is more effective 
than a simple reinforcement of the weak eye (Xu, He, & Ooi, 2010). 

Some examples of recovery from deprivation have been reported previously, 
hinting at residual plasticity in human adults. On late removal of unilateral cata- 
racts (Ellemberg, Lewis, Maurer, Brar, & Brent, 2002), some visual recovery was 
observed, but mainly involving higher cognitive functions such as global motion 
perception, probably mediated by associative cortex, rather than basic visual sensi- 
tivity mediated by primary visual cortex. On the other hand, preserved visual pars- 
ing is observed after late removal of bilateral congenital cataracts (Fine, Smallman, 
Doyle, & MacLeod, 2002; Ostrovsky, Meyers, Ganesh, Mathur, & Sinha, 2009), this 
is in line with evidence showing that the effects of binocular deprivation (which 
does not drive neural competition) are less severe and less durable than those of 
monocular deprivation (Wiesel & Hubel, 1965a). 
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A few recent experimental studies have shown that short binocular deprivation 
in adults can reveal some residual neural plasticity in human vision. For example, 
Kwon, Legge, Fang, Cheong, & He (2009) showed a slight improvement in contrast 
sensitivity thresholds after four hours of contrast reduction, correlated with an in- 
creased BOLD signal in V1 and V2. Boorojerdi et al (2000) observed an increase in 
excitability of primary visual cortex (TMS phosphene thresholds decreased and 
BOLD signal in V1 was enhanced) after a few hours of binocular blindfolding, con- 
firming the important role of intracortical inhibition and excitation balance for plas- 
ticity (Boroojerdi et al., 2000). This was also supported by results from the same lab 
demonstrating that benzodiazepine administration completely annuls the effect of 
light deprivation on cortical excitability (Boroojerdi, Battaglia, Muellbacher, & Co- 
hen, 2001). Zhang, Bao, Kwon, He, & Engel (2009) showed that four hours of selec- 
tive attenuation of a specific orientation improved slightly discrimination thresholds 
of the deprived orientation. 

In our current study, binocular rivalry, which probes neural, inhibition- 
generated competitive mechanisms (Blake & Logothetis, 2002; Klink, Brascamp, 
Blake, & van Wezel, 2010; Levelt, 1966; Tong, Meng, & Blake, 2006), revealed that 
even ocular dominance, thought to be plastic only during the critical period, has 
considerable residual plasticity in young human adults. The effect of monocular 
deprivation that we found on binocular rivalry shares some characteristics with con- 
trast adaptation, such as the exponential decay (Wark, Fairhall, & Rieke, 2009). 
However, the effects described here are far more long-lasting than those reported for 
adaptation. Bao & Engel (2012), for example, found that 15 minutes of de- 
adaptation cancelled the effects of 4 hours of contrast adaptation, whereas our effects 
persisted for over three hours, longer than the deprivation period, implicating plas- 
ticity mechanisms other than those affected by contrast adaptation. The effects may 
well be related to contrast adaptation, but have characteristics quite different from 
those reported to date, engaging plastic changes in neural activity that are far more 
long-lasting than previously described. 

Our results show that monocular deprivation had more dramatic consequences 
on the dynamics of binocular rivalry when chromatic- rather than luminance- 
modulated gratings were tested. Equiluminant gratings are known to reduce the re- 
sponse of M-cells in favor of P-cells, which are sensitive to chromatic differences 
(Hubel & Livingstone, 1990; Schiller & Malpeli, 1978). Our results suggest that the 
parvo-pathway is more susceptible to monocular deprivation in adult humans, as 
monocular deprivation produced longer-lasting effects, with a slower decay, for 
chromatic than luminance gratings. These results suggest that the parvo system is 
affected by monocular deprivation for longer periods compared with the magno sys- 
tem, pointing to a higher degree of plasticity. The hypothesis of a leading role of the 
parvo system in mediating the effect of Monocular Deprivation on Binocular Rivalry 
is in line with evidence showing that during the critical period, monocular depriva- 
tion has more severe effects on the parvo system, with ocular dominance columns 
shrinkage of macaque primary visual cortex being larger in layer IVch (Horton & 
Hocking, 1997). Consistent with this evidence, in humans, visual features associated 
with the magno-system (such as motion perception) are more resistant to visual 
deprivation, showing spared functions after recovery from blindness (Fine et al., 
2003; Maurer, Lewis, & Mondloch, 2005; Ostrovsky, Meyers, Ganesh, Mathur, & 
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Sinha, 2009), indicating that the parvo system is in general more vulnerable to the 
effects of visual deprivation. 

Different neural functions, even within the same sensory system, may develop at 
different rates and have different critical periods. There appears to be a link between 
the developmental time-course of the different visual functions and their vulnerabil- 
ity to abnormal visual experience. The “Detroit Model” of Levi (2005) proposes that 
visual functions that develop slowly are more sensitive to the effects of sensory dep- 
rivation (i.e. retain a higher degree of experience-dependent plasticity), following the 
principle “last-hired, first-fired” (Levi, 2005). Achromatic and chromatic vision have 
different developmental timecourses, the first developing fast, the other being a late 
bloomer in visual development, with VEPs to chromatic stimuli developing much 
later than those to luminance (Morrone, Burr, & Fiorentini, 1990), and not becom- 
ing adult-like until 12-14 years of age in humans, and latencies not completely ma- 
ture until 17-18 years of age (Crognale, 2002). As chromatic parsing is mostly asso- 
ciated with P-cell activity (Gegenfurtner & Kiper, 2003; Hubel & Livingstone, 1990; 
Schiller & Malpeli, 1978), the late development of the chromatic vision in humans 
suggests that P-cells retain a high degree of plasticity even after the closure of the 
critical period, our results confirmed this spared plasticity. 

One interesting point of our results is that monocular deprivation affected rival- 
ry between chromatic and luminance grating in a different way, having more impact 
on sustained rivalry for equiluminant grating. Onset and sustained rivalry show dif- 
ferent characteristics (reviewed in Stanley, Forte, Cavanagh, & Carter, 2012): for ex- 
ample, onset rivalry shows a stable and predictable individual bias that varies across 
the visual field according to the zones of monocular dominance and is therefore 
linked to (although not totally explained by) ocular dominance, while a hallmark of 
sustained rivalry is the unpredictability of the perceptual switches (for accounts on 
perceptual memory and onset rivalry see also de Jong, Knapen, & van Ee, 2012; 
Noest, van Ee, Nijs, & van Wezel, 2007; Pastukhov & Braun, 2008). In general, onset 
rivalry has been shown to be more sensitive to early visual features than sustained 
rivalry, for example, small imbalances in contrast and luminance between stimuli 
strongly affect onset rivalry leaving sustained rivalry almost unchanged. Equating 
the strength of the rivalring images does not annul the stable and consistent bias 
shown by every observer at the onset of rivalry while balancing stimulus strength 
equates sustained rivalry dominance (Stanley, Carter, & Forte, 2011). Because of the 
differences between onset and sustained rivalry, it is likely that the two phenomena 
are mediated by different mechanisms. The hypothesis of a different contribution of 
M- and P-pathway to the two types of rivalry, is in line with recent evidence (Den- 
ison & Silver, 2012) showing that M-stream is more involved in eye-rivalry, while 
the P-stream is more engaged in stimulus-rivalry. Stimulus rivalry is a particular 
form of binocular rivalry revealed by the interocular-switching (IOS) paradigm first 
proposed by Logothetis, Leopold, & Sheinberg (1996) in which rivalring images are 
swapped between the eyes 3 times per second and can lead both to rapid-regular 
switches (eye rivalry) or slow-irregular switches (stimulus rivalry). The finding that 
monocular deprivation had different effects on binocular rivalry of chromatic and 
luminance gratings, suggests that onset and sustained rivalry involve different neural 
mechanisms and possibly a different participation of the Magnocellular and Par- 
vocellular visual pathway. Our results also suggest a different contribution of M- and 
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P-pathway to the two types of rivalry, the P-pathway playing a major role in mediat- 
ing sustained rivalry rather onset rivalry. 

In our previous brief report (Lunghi, Burr, & Morrone, 2011), we showed that 
monocular deprivation also influenced apparent contrast, with stimuli presented to 
the deprived eye appearing on average 36% higher in contrast than stimuli presented 
to the non-deprived eye. The effect of deprivation on the dynamics of binocular ri- 
valry could not be explained by the boost in apparent contrast because in order to 
affect mean phase durations in a way similar to deprivation, contrast in one eye had 
to be higher by a factor of 3. We therefore speculated that short-term monocular 
deprivation acted by increasing contrast gain of the deprived eye as a first attempt of 
the visual system to compensate for the lack of information. The fact that monocular 
deprivation had more severe consequences for equiluminant stimuli could reflect the 
different contrast gains of M- and P-cells. While M-cells responses rapidly saturate 
for stimuli above 20% of contrast, most P-cells do not show saturating responses to 
chromatic stimuli even at high chromatic contrasts (Purpura, Kaplan, & Shapley, 
1988; Solomon & Lennie, 2005). If monocular deprivation increases contrast gain of 
the deprived eye, it is likely to have a greater effect on P-cell responses rather than on 
M-cells, which are limited by saturation. 

The gamma-like shape of phase durations distributions has been considered a 
hallmark of binocular rivalry and bistable perception in general (Carter & Pettigrew, 
2003; van Ee, 2005). However the two parameters defining the gamma distribution 
usually correlated, and are consequently considered redundant (De Marco, Penengo, 
& Trabucco, 1977; Mamassian & Goutcher, 2005). One last interesting result from 
our data is that monocular deprivation disrupted the correlation between the two 
parameters defining the gamma distribution used to fit phase durations distributions 
of the two eyes, even though the significance of this finding is uncertain. 


3.3.4. Conclusions 


In conclusion, we have shown that a brief period of monocular deprivation has 
drastic consequences on visual perception that are likely to reflect neuroplastic 
changes at the level of the primary visual cortex. We also showed that the perceptual 
bias of binocular rivalry showed a much slower decay for chromatic than for lumi- 
nance gratings, lasting for at least 180 minutes after removal of the eye-patch. As 
equiluminant gratings are known to reduce response of M-cells, our results suggest 
that P-cells are more susceptible to the effect of visual deprivation and retain a high- 
er degree of residual experience-dependent plasticity.That the adult visual system 
retains a high degree of experience-dependent plasticity is important for under- 
standing neural reorganization following late visual loss and for reconsidering sensi- 
tive periods in human vision. Binocular rivalry revealed itself as a sensitive probe for 
neuro-plastic changes and could be a useful non-invasive tool for monitoring plastic 
changes during occlusion therapy for amblyopia. 


3.4. Summary and Conclusions. 


We have demonstrated clear functional changes of adult human vision lasting 
up to 180 minutes after brief periods of deprivation of pattern vision to one eye. Fol- 
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lowing 180 minutes of monocular deprivation of pattern vision changed dramatical- 
ly the dynamics of binocular rivalry, causing the deprived eye to dominate conscious 
perception over the non-deprived eye in adult humans. This paradoxical result indi- 
cates that the visual system initially reacts to monocular deprivation by homeostati- 
cally boosting activity of the deprived eye in order to compensate for the lack of in- 
formation caused by deprivation. 

We propose that the neural changes underlying this form of monocular- 
deprivation-driven neuroplasticity is an up-regulation of contrast gain control 
mechanisms, hypothesis that is supported by the evidence that after monocular dep- 
rivation images presented to the deprived eye appeared to be higher in contrast 
compared to images presented to the non-deprived eye. 

We have also shown that the effects of monocular deprivation are more long- 
lasting (the duration of the effects increases by a factor of 2) when equiluminant 
gratings are tested compared to luminance-modulated gratings. This result suggests 
a higher vulnerability of chromatic vision to the effect of monocular deprivation and 
possibly a higher degree of residual plasticity for the Parvocellular pathway. 

Taken together, these results call for a re-evaluation of adult visual cortical plas- 
ticity, fundamental to understanding how the adult visual system reacts to sensory 
loss, with implications for developing therapeutic strategies that exploit the intrinsic 
plasticity of the visual cortex. 
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Chapter 4 
Binocular Rivalry as a tool to monitor visual system plasticity dur- 
ing occlusion therapy in amblyopic children 


4.1. Introduction 


We have demonstrated that the adult human visual system retains a degree of 
neuroplasticity higher than previously thought: 150 minutes of monocular depriva- 
tion using a translucent eyepatch had important perceptual consequences, favouring 
dominance of the deprived eye during binocular rivalry (Lunghi, Burr, & Morrone, 
2011), with a significant effect being measurable up to 180 minutes following 
eyepatch removal, depending on the type of visual stimuli used. 

Plasticity is a fundamental property of the developing visual brain, allowing dy- 
namic adaptation of the young organism to the environment. Taking advantage of 
the incredible plastic potential of young children’s visual system, the most classic 
treatment for amblyopia (that is the most common cause of visual impairment in 
children) is occlusion therapy, consisting in patching the fellow eye in order to force 
the brain to use the weaker amblyopic eye reinforcing its connections. 

Having demonstrated that binocular rivalry is a sensitive probe for visual plastic- 
ity following monocular occlusion, we were interested in investigating the perceptual 
consequences of prolonged monocular deprivation on rivalry dynamics, we there- 
fore tested binocular rivalry during occlusion therapy (that requires patching of the 
fellow eye over extended periods of time) in amblyopic children. Data recorded so 
far are still preliminary, but they are encouraging, suggesting that binocular rivalry 
can be used to monitor visual plasticity during occlusion therapy. 


4.1.1. Amblyopia 


Amblyopia is defined as a developmental disorder of spatial vision in which a 
deficit in visual acuity is present despite an apparently physically normal eye, a defi- 
cit that is not optically correctable and is commonly known as “lazy eye” syndrome. 
Amblyopia is usually associated with strabismus, anisometropia and early visual 
deprivation (i.e. early onset or congenital cataracts), Ciuffreda, Levi, & Selenow, 
1991; Levi & Li, 2009, and is one of the most frequent causes of visual loss in infants 
and young children, affecting 1-5% of the human population (Webber & Wood, 
2005). Amblyopia severity correlates with the onset age of the disease, that appears 
to have a “sensitive period” spanning from the first year of life to 6-8 years of age in 
humans (D. H. Baker, Meese, & Hess, 2008; Maurer, Lewis, Brent, & Levin, 1999; 
Maurer, Lewis, & Mondloch, 2005). The onset of the sensitive period for amblyopia 
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seems to be strictly associated with the development of binocular interactions in 
primary visual cortex and with the development of visual sensitivity to high spatial 
frequencies that are more susceptible to the blur caused either by high refractive er- 
rors or by cataracts (Kiorpes, Kiper, O'Keefe, Cavanaugh, & Movshon, 1998); the 
end of the sensitive period for amblyopia correlates with the end of the critical peri- 
od for the same features. 

The severity of the visual loss associated with amblyopia increases with increas- 
ing imbalance between the eyes (anisometropia) rather than for binocularly equiva- 
lent refractive errors (isometropia) because of the competitive interactions between 
the eyes in establishing and reinforcing the neural connections with the visual cortex 
(McKee, Levi, & Movshon, 2003; Weakley, 2001). Isoametropic refractive patients in 
fact do not develop amblyopia and show much less severe pattern of visual loss and 
spared stereoacuity compared to anisometropes (Levi, McKee, & Movshon, 2011). 
Binocular blur appears therefore to be more tolerated by the visual system than mo- 
nocular blur; if not treated early after birth, in fact, monocular cataracts provoke se- 
vere amblyopia (for review see (Maurer, Lewis, & Mondloch, 2005). 

One of the important factors in determining the pattern of visual loss associated with 
amblyopia is binocularity, which is totally devastated in strabismic amblyopes. Anisome- 
tropic patients lacking of binocular function in fact show patterns of visual deficits simi- 
lar to those of strabismic patients. Deficits in contrast sensitivity, spatial resolution (acui- 
ty) and stereoacuity associated with anisometropia have been thoroughly investigated by 
Levi, McKee and Movshon (Levi, McKee, & Movshon, 2011). They demonstrated that 
acuity is impaired with increasing anisometropia, with strabismus having a multiplica- 
tive effect (factor of 2.5) on spatial acuity, while contrast sensitivity has been shown to be 
less sensitive to anisometropia severity, that is (as anticipated above) instead crucial for 
stereopsis. In general, anisometropes lacking of binocularity show severe amblyopia and 
a longer prognosis with treatment (Kivlin & Flynn, 1981). 


4.1.2. Occlusion Therapy 


The most common and traditional treatment for amblyopia is the occlusion 
therapy, that consists in patching the fellow eye for an amount of hours per day 
ranging from 2 hours to full time depending on the severity of the disease (i.e. the 
degree of anisometropia). The idea behind the occlusion therapy is to force the pa- 
tient to use the “lazy eye”, obliging the brain to use the weaker connections from the 
amblyopic eye with the aim of reinforcing them. The success of the treatment de- 
pends on the patching dose, on whether or not the patient shows spared binocular 
function, on the presence of strabismus, the severity of amblyopia and the age of the 
patient (Stewart, Moseley, Stephens, & Fielder, 2004). The dose-response of the oc- 
clusion therapy shows a long time constant, with acuity of the amblyopic eye im- 
proving of about 26% for every 120 hours of treatment and requires therefore patch- 
ing over extended periods of time (from a few months up to one year or more). 

Since sensitivity to high spatial frequency is a crucial factor for the development 
of amblyopia, during the past decade a less aggressive form of occlusion has been 
proposed for the treatment of amblyopia that consists in applying Bartenger foils on 
the spectacle lens of the fellow eye rather than using an opaque eyepatch (a too ag- 
gressive patching could be in fact detrimental provoking a visual loss of the fellow 
eye). Bartenger foils (Ryser Optik, Gallen, Switzerland) are translucent diffuser filters 
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made of scattering micro-elements designated to reduce visual acuity by different 
levels, depending on the strength of the filter. Bartenger foils act by attenuating the 
high and mid-range of spatial frequencies without showing spurious resolution and 
phase shifts (Perez, Archer, & Artal, 2010), and have been demonstrated to be effec- 
tive for the treatment of moderate amblyopia (Iacobucci, Archer, Furr, Martonyi, & 
Del Monte, 2001). 

Even though occlusion therapy is thought to be maximally effective if adminis- 
tered during the sensitive period for amblyopia (so within the first 6-8 years of age) 
there is some evidence showing residual plasticity of the amblyopic visual system of 
older children and young adults, even though a more aggressive treatment is re- 
quired to obtain some improvement in young adults (Kupfer, 1957). More evidence 
in favour of a residual plastic potential of the adult amblyopic visual system is ob- 
tained from studies showing an improvement of the amblyopic eye acuity following 
a visual loss of the fellow eye (macular degeneration, El Mallah, Chakravarthy, & 
Hart, 2000, refractive errors, Rahi et al., 2002). The loss of acuity in the fellow eye is 
thought to unmask the connections from the amblyopic eye that are likely to be sup- 
pressed and not destroyed. 


4.2. Binocular Rivalry during occlusion therapy 


We have previously demonstrated that following a brief period of monocular 
deprivation (150 minutes), visual perception of adults during binocular rivalry is 
strongly biased in favour of the deprived eye that wins the struggle for visual aware- 
ness over the non-deprived eye (Lunghi, Burr, & Morrone, 2011). Our results are 
apparently in contrast with the outcome of occlusion therapy used to treat amblyo- 
pia, because we showed, counter intuitively, a strengthening of the deprived eye fol- 
lowing monocular patching. However, binocular rivalry is a peculiar phenomenon, 
whose dynamics are not necessarily linked with the mechanisms underlying neural 
plasticity mediating the recovery of visual acuity of the amblyopic eye observed dur- 
ing the occlusion therapy. It is therefore interesting to test binocular rivalry during 
the occlusion therapy in amblyopic patients in order to investigate the relationship 
between the perceptual bias observed in adults after monocular deprivation with the 
recovery of amblyopia during monocular deprivation of the fellow eye. 

Another difference between our study and occlusion therapy is the duration of 
monocular deprivation: we were in fact only able to test short-term deprivation (150 
minutes) and we cannot exclude therefore the involvement of different neural mech- 
anisms after prolonged patching. During occlusion therapy, patients are required to 
wear the Bartenger filter for an extended period of time (several months), allowing 
to test the effect of monocular occlusion on the dynamics of binocular rivalry after 
prolonged eye-patching, that would be otherwise impossible with normal observers. 


4.2.1. Materials and Methods 
Subjects 
Five children (one female), mean age 5.8+0.42, participated in the experiment. 


Children aged between 5 and 7 years were selected because of the developmental 
timecourse of binocular rivalry (Hudak et al., 2011; Kovacs & Eisenberg, 2005). All 
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of them were anisometropic amblyopes at the first examination after the prescrip- 
tion of the spectacles. All had spared binocularity (Lang test) and didn’t show stra- 
bismus. None of them had been previously treated with occlusion therapy. The ex- 
periment was performed according to the principles of the Declaration of Helsinki 
and was approved by the ethics committee of the hospital (Azienda Ospedaliaro- 
Universitaria Meyer, Florence). Children were accompanied by their parents, who 
were also present during the test. Written informed consent was obtained from the 
participants’ parents. 


Apparatus and Stimuli 

The experiment was set up in the outpatients’ ophthalmology department at the 
hospital Meyer in Florence. The room was dark and quiet. Participants were first 
tested for visual acuity (Snellen acuity) of their two eyes using vector Snellen Charts 
for children. Table 3.4.1 reports children’s visual acuity measured at different time 
intervals from the beginning of the occlusion therapy. 

Visual stimuli were generated by a portable VSG 2/5 (CRS, Cambridge Research 
Systems) housed on a laptop (DELL) and controlled by Matlab programs, they were 
displayed on a gamma-corrected CRT monitor (MODEL) driven at a resolution of 
1024X600 pixels, with a refresh rate of 120 Hz. Observers’ head was stabilized with a 
chinrest placed at a distance of 57 centimeters from the monitor, participants viewed 
the visual stimuli through CRS FE-Shuttering Goggles that were fastened to the 
chinrest. The goggles were synchronized with the monitor refresh rate through the 
VSG 2/5 and occluded alternatively the two lenses at each frame allowing dichoptic 
viewing. 

Visual stimuli were achromatic orthogonal (horizontal and vertical) Gabor 
patches (size 2°, spatial frequency 2 cpd, contrast 75%) presented on a uniform aver- 
age grey background (luminance 37 cd/m’) in central vision, with a black fixation 
point and a black common squared frame (size 2.5°). Observers reported verbally 
their visual perception and responses were recorded through the computer keyboard 
manipulated by the experimenter. 

Occlusion therapy was achieved by sticking a Bartenger 0.4 filter on the spectacle 
lens of the fellow eye. Since the optical features of the Bartenger foils are not homo- 
geneous and often differ from the labeled density designation (Perez, Archer, & Ar- 
tal, 2010), each filter was first controlled by the experimenter for the correct level of 
blur by measuring Snellen acuity through it and was then applied on the appropriate 
spectacle lens. 


Task and Procedures 

To make the setup and the task appealing for children, we covered the monitor 
with a black cloth with yellow stars pinned on it and we stuck some cartoon charac- 
ters on the monitor in order to make it look like a sort of “magic box”. A picture of 
the setup is reported in Figure 4.1. We invented a story, that we told to the children 
showing a Power Point presentation, about a magic competition and we asked the 
children to be the referees of the context, telling them that the magic tricks could on- 
ly be seen through the goggles. Children were very keen to perform the task, some of 
them even enthusiastic. They were trained to report verbally whether the “stripes” 
(Gabor patches) on the monitor were “standing up” (vertical) or “lying down” (hori- 
zontal). The experimenter held the appropriate key of the computer keyboard ac- 
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cording to the visual perception reported by the observer. Children were motivated 
to perform accurately the task because at the end of each 3-minutes experimental 
block they were asked to judge which of the characters had won the magic context 
(one was associated with the horizontal, the other with the vertical orientation). The 
orientation presented to each eye was swapped at every session as well as the orienta- 
tion associated with a particular character, in order to reduce the possibility of re- 
sponse bias in favour of one or the other orientation. 

During the first training session binocular rivalry was simulated by presenting 
the same orientation to both eyes and changing it at random intervals mimicking the 
dynamics of binocular rivalry. During the training session the experimenter could 
check the accuracy of the child and ended the trial when the observer reported the 
visual orientation precisely. Children that were not able to perform the task were 
discarded and did not take part to the binocular rivalry experiment. 

After the training session a short (90 seconds) binocular rivalry session was rec- 
orded to determine the quality of binocular rivalry for the different children. Ob- 
servers that did not show binocular rivalry (either not alternating at all or fusing the 
two visual images reporting to perceive a plaid) were discarded. None of the observ- 
ers that we considered for analysis reported periods of patchy rivalry. Binocular ri- 
valry was measured before the placement of the Bartenger foil on the lens of the fel- 
low eye (baseline measurements) and at different time intervals following the onset 
of the occlusion therapy: 120 minutes, 30 days, 60 days, 150 days. 2x180 seconds ex- 
perimental blocks were recorded for each interval. 


Figure 4.1. Experimental Setup 

Two pictures of the child-friendly experimental setup are shown. The child sat in front of the 
monitor, the head was stabilized by a chin and forehead rest and the shuttering goggles were 
attached to it. The monitor was covered with a black cloth and yellow stars and a “sorcer 
Mickey Mouse” cartoon was stuck in correspondence of the visual stimulus. Children were 
asked to help Mickey Mouse in judging the magic context and were told that the goggles were 
especially constructed to show magic tricks. 


4.2.2. Preliminary results 


The outcome of the occlusion therapy is shown in Table 4.1, where visual acuity 
of the two eyes is reported for the different time intervals tested. All children showed 
an improvement of visual acuity of the amblyopic eye, improvement that, for some 
children, was virtually complete two months after the beginning of the treatment 
and stabilized 5 months after. If binocular rivalry dynamics were governed by the 
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same neural mechanisms underlying the recovery of amblyopia we would expect to 
observe an initially strong perceptual bias in favour of the fellow eye (that we will de- 
fine as the deprived eye because it is the one that will undergo the patching proce- 
dure) and a subsequent reduction of the bias, with balance between the eyes being 


restored with recovered acuity of the amblyopic (non-deprived) eye. 


Subject | Date of Initial Visus Visus after Visus Visus after 
Birth 1 month 2 months 5 months 
S1 18-08-06 | RE 0.9 - LE 0.6 | ----------------- RE 0.9 - LE 0.9 | RE 1.0 - LE 1.0 
S3 26-02-05 | RE 0.4- LE 1.0 | ----------------- RE 0.7 - LE 1.0 | RE 0.8 - LE 1.0 
S6 12-06-06 | RE 1.0- LE 0.5 | RE 1.0 -LE 0.7 | RE 1.0 - LE 0.9 | ----------------- 
S7 12-07-05 | RE 1.0- LE 0.4 | RE 1.0- LE 0.8 | RE 1.1 - LE 0.9 | ----------------- 
S8 06-04-04 | RE 1.0- LE0.6 | RE 1.0 -LE 0.7 | RE 1.0 - LE 0.8 | ----------------- 


Table 4.1. Participants’ age and visual acuity in the two eyes. 

The table reports the date of births of the five children tested and the visual acuity of the right 
and left eyes (RE and LE) measured using Snellen tables. All children show an improvement 
of visual acuity of the amblyopic eye following occlusion therapy. The improvement is al- 
ready important 1 month after the onset of the treatment. 


The proportion of time spent by the observers seeing the stimulus presented to the 
deprived (fellow) eye for the different time intervals recorded is reported in Figure 
4.2. The dashed line represents perfect balance between the two eyes (with domi- 
nance proportion being 50%). In the baseline measurements, surprisingly, all but 
one observer showed balanced rivalrous perception, despite anisometropia. We 
chose visual stimuli having an optimal spatial frequency (2 cycles per degree of visu- 
al angle), that should be well visible even for the amblyopic eye, the blur caused by 
amblyopia, in fact, principally affects high spatial frequencies. Nevertheless, this re- 
sult indicates that the dynamics of binocular rivalry are not affected by the weaken- 
ing of the signal from the amblyopic eye provoked by the pathology, suggesting that 
binocular rivalry engages different neural mechanisms. 

Equally surprising is the result that the occlusion of the fellow eye disrupted the 
initial balance, with visual perception during binocular rivalry being biased in favour 
of the deprived (fellow) eye. The measurements recorded 2 hours after the onset of 
the occlusion therapy confirm our previous findings obtained with adult normal ob- 
servers undergoing monocular patching (Lunghi, Burr, & Morrone, 2011). Interest- 
ingly, the bias in favour of the fellow (deprived) eye is maximum 30 days following 
the onset of the treatment, with the deprived eye dominating competition with the 
non-deprived (amblyopic) eye 80% of the time, despite the recovered acuity of the 
amblyopic eye compared to the baseline measurements. 60 days after eye-patching, 
the recovery of the amblyopic eye is almost complete and balance between the two 
eyes during binocular rivalry is restored, balance that is confirmed in measurements 
recorded 150 days after the onset of occlusion therapy. 
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Figure 4.2. Proportion of Dominance in favour of the deprived eye. 

The proportion of time spent seeing the visual stimulus presented to the deprived (fellow) eye 
is plotted as a function of time elapsed from the onset of the occlusion therapy (temporal 
units on the x scale are arbitrary). The dashed line represents perfect balance between the 
eyes. The small hollow symbols stand for individual observers’ performance, while the full 
black stars are the average (error bars represent s.e.m). Despite anisometropia, the baseline 
measurements show a balanced visual perception during binocular rivalry. Following mo- 
nocular patching with Bartenger 0.4 filters, visual perception is biased in favour of the de- 
prived eye, the bias is already present 2 hours following patching and is maximum 30 days 
after. 60 days following the beginning of treatment, balance between the eyes is re-established 
and reinforced 150 days after, when the amblyopic eye has virtually completely recovered (see 
Table 3.4.1). 


4.2.3. Discussion 


The data presented above are still preliminary and incomplete, and much more 
work needs to be done, but nevertheless, they show an interesting trend: the dynam- 
ics of binocular rivalry recorded before and during occlusion therapy seem to be un- 
related with the visual loss provoked by amblyopia and the subsequent recovery of 
visual acuity during the treatment. Before the onset of the therapy, in fact, all observ- 
ers tested had an important anisometropia between the eyes, with visual acuity being 
perfect in the fellow eye and impaired in the amblyopic eye. Nevertheless, surpris- 
ingly, observers’ perception during binocular rivalry was balanced, with only one 
child showing a strong perceptual bias in favour of the fellow eye. 

During occlusion therapy visual acuity of the amblyopic eye improved and was 
virtually recovered two months following the beginning of the treatment. Interest- 
ingly, during the first month of treatment, that is when plasticity is maximum and 
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the amblyopic eye rapidly improves visual acuity, the initial perceptual balance be- 
tween the eyes was disrupted and visual perception during binocular rivalry was 
strongly biased in favour of the fellow (deprived) eye, that dominated 80% of the 
time over the amblyopic (non-deprived) eye. The initial perceptual balance between 
the eyes was then re-established two months following the onset of the treatment 
and reconfirmed five months after, when visual acuity of the amblyopic eye was sta- 
bilized and neural-plasticity probably lowered. 

The marked dissociation between the strength of the visual signals (diminished 
by amblyopia and then recovered during the treatment) and visual perception dur- 
ing binocular rivalry suggests that different neural mechanisms underlie the two 
phenomena. Binocular rivalry is a peculiar case of bistable vision that engages com- 
petition between the monocular inputs and the neural representation of the rivalring 
visual stimuli at different stages of neural processing (for review see: Tong, Meng, & 
Blake, 2006). Importantly, binocular rivalry also engages strong adaptation and re- 
ciprocal inhibition between the eyes (for review see: Alais, 2012) and is in principle 
sensitive to the level of intracortical inhibition of the visual cortex. 

Balance between intracortical inhibition and excitation at the level of the visual 
cortex is crucial for experience-dependent plasticity. As discussed in the introduc- 
tion (3.1), the development of GABA-ergic inhibition is one of the most relevant fac- 
tors provoking the closure of the critical period in mammals (Fagiolini & Hensch, 
2000), and plasticity in adult animals can be restored by decreasing pharmacologi- 
cally the levels of intracortical inhibition (Harauzov et al., 2010; Maya Vetencourt et 
al., 2008). Finally, GABA-ergic neurons are the only cells in the visual cortex retain- 
ing some form of experience-dependent plasticity (Hendry & Jones, 1986; Lee et al., 
2006). If experience-dependent plasticity driven by the occlusion therapy causes 
some changes in intracortical inhibition (probably lowering the level of GABA-ergic 
inhibition), the perceptual changes observed in binocular rivalry are likely to reflect 
the timecourse of neural plasticity during the treatment of amblyopia. 

Even though our data are still preliminary and the interpretation speculative so 
far, the correlation between visual acuity recovery and the perceptual bias in favour 
of the fellow eye that we observed is striking and suggests that binocular rivalry 
could be an useful non-invasive tool to monitor the efficiency and timecourse of oc- 
clusion therapy for amblyopia and maybe predict its outcome. 
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Chapter 5 
General Conclusions 


In this work we have revealed some unexpected properties of the early visual 
cortex. We have challenged the view that considers primary sensory areas as being 
exclusively dedicated to unisensory processing by showing that touch can interact 
with vision during binocular rivalry at early stages of visual analysis, probably al- 
ready in V1. Our results have further challenged the notion of neuroplasticity being 
restricted to a sensitive period early in life by demonstrating that a brief period of 
monocular deprivation in adult observers provokes important perceptual effects that 
results in unbalancing visual perception during binocular rivalry in favour of the de- 
prived eye with an effect being measurable for a period of time exceeding the dura- 
tion of deprivation. 

We propose that using binocular rivalry as psychophysical method to investigate 
these aspects of visual perception is the crucial factor that allowed the discovery of 
these important properties of early visual cortices. Binocular rivalry, in fact is a pecu- 
liar visual phenomenon presenting unique characteristics that can be used to un- 
mask subtle behavioural effects. For example, the intrinsic ambiguity of visual rival- 
rous perception can help in unmasking signals from other sensory modalities, that 
are normally concealed by a strong and reliable visual input, without needing to 
weaken visual stimulation by degrading the visual stimuli. In addition, since binocu- 
lar rivalry requires the interaction between monocular signals, it is resolved at the 
cortical level, that is the first stage of neural processing in which the monocular sig- 
nals are integrated. Using binocular rivalry therefore ensures that the cortical origin 
of the neural processes that are investigated. 

One of the most stryking features of binocular rivalry is the dissociation between 
constant physical stimulation and alternating visual perception: albeit two different 
visual stimuli are contemporaneously displayed on the retinae, only one of them is 
consciously perceived at a time, the other being suppressed from visual awareness. 
While the neural signal associated with the dominant visual stimulus is processed by a 
complex hierarchy of cerebral areas, starting from primary visual cortex and reaching 
object-sensitive areas in the infero-temporal cortex, the neural trace of the suppressed 
visual stimulus is not treacable outside of V1 or V2. Factors interfering with the sup- 
pressed visual signals are therefore extremely likely to reflect neural interactions occur- 
ring at these early stages of visual processing. This aspect of binocular rivalry im- 
portantly revealed the early interaction between vision and touch that we found. 

Binocular rivalry engages strong competition between the monocular signals 
and is therefore a sensitive probe for measuring the relative strength of the signal as- 
sociated with each eye. Other psychophysical methods, in fact, involve collaborative 


Claudia Lunghi, Early cross-modal interactions and adult human visual cortical plasticity revealed by binocular rivalry 
ISBN 978-88-6655-686-2 (print) ISBN 978-88-6655-687-9 (online), CC BY 4.0, 2014 Firenze University Press 


Early cross-modal interactions and adult human visual cortical plasticity 


rather than competitive interactions between the eyes. Binocular rivalry is a unique 
method to separately track the strength of the monocular signals in a context in 
which they compete with each other. This is the property of binocular rivalry that 
was crucial for revealing the effects of monocular deprivation on adult visual percep- 
tion, for competition between the monocular signals is one of the important factors 
underlying neuroplastic changes in the visual cortex. 

Moreover, visual competition during binocular rivalry is thought to be driven by 
reciprocal inhibition between the monocular signals. Binocular rivalry could there- 
fore be used as a sensitive method for indirectly evaluate the balance between intra- 
cortical inhibition and excitation, the dynamics of binocular rivalry being regulated 
by inhibitory processes. Balance between inhibition and excitation at the cortical 
level is crucial for neuroplasticity, for either increasing excitation or decreasing in- 
tracortical inhibition restores neuroplasticity in the visual cortex of adult animals 
and critical periods depends on the development and maturation of GABAergic in- 
hibition. This characteristic is probably the reason why we have shown that binocu- 
lar rivalry can be efficiently used as a non-invasive tool to monitor neuroplastic 
changes in the visual cortex of amblyopic children during the recovery of visual acui- 
ty in the amblyopic eye caused by the occlusion of the fellow eye. 

In conclusion, taking advantage of the peculiar features of binocular rivalry de- 
tailed above, we have been able to reveal novel properties of early visual cortex such 
as residual plasticity and early cross-modal processing. 

Taken together, our results have important implications for understanding the 
consequences of sensory loss on the adult visual system, suggesting that the recruit- 
ment of primary visual cortex for the processing of information from other sensory 
modalities observed in blind patients occurs by unmasking pre-existing connections 
from those modalities to the primary visual cortex, rather than in creating them ex- 
novo. 

Finally, our deprivation studies also show that the adult visual cortex retains a 
higher degree of residual experience-dependent plasticity than previously thoutght. 
These results strongly suggest that the first response of the adult visual system to 
deprivation is a compensatory homeostatic boost of neural activity, probably occur- 
ring through up-regulation of contrast gain control mechanisms, as a first attempt to 
compensate for the impoverished incoming information provoked by visual depriva- 
tion. 
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